Adiciona possibilidade de uso com MongoDB #278

mbnunes · 2025-01-29T13:00:12Z

Adicionado o uso do MongoDB no projeto. Ainda falta terminar as documentações

cuducos

Revisei apenas a parte de integração (ou seja, ainda não olhei para a implementação do MongoDB em si, db/mongo.go e seus testes), mas muito bom! Feliz demais com essa colaboração!

Deixei alguns comentários, tem coisas que ainda não precisamos mudar antes de dar merge, mas tá muito bom!

.env.sample

cmd/api.go

cmd/cmd.go

cmd/transform.go

docker-compose.yml

transform/venues.go

transform/transform.go

cuducos

Mais comentários, ainda sem chegar no módulo do MongoDB, ainda focando na integração : )

.env.sample

cuducos · 2025-01-30T15:30:52Z

cmd/api.go

+		uri := os.Getenv("DATABASE_URL")
+		if strings.HasPrefix(uri, "mongodb://") {
+			mdb, _ := db.NewMongoDB(mongoDatabase)
+			if err != nil {
+				return err
+			}
+
+			defer mdb.Close()
+			api.Serve(&mdb, port, nr)
+
+		} else if strings.HasPrefix(uri, "postgres://") || strings.HasPrefix(uri, "postgresql://") {
+			pg, err := db.NewPostgreSQL(u, postgresSchema, nr)
+			if err != nil {
+				return err
+			}
+			defer pg.Close()
+
+			api.Serve(&pg, port, nr)
+		} else {
+			return fmt.Errorf("unsupported database URI, must start with postgres:// or mongodb://")
+		}


Bem melhor : ) Mas ainda dá para evitar repetição usando a interface, algo como:

var db database } else if strings.HasPrefix(uri, "postgres://") || strings.HasPrefix(uri, "postgresql://") { db = postgres{DATABASE_URL} } else if strings.HasPrefix(uri, "mongodb://") { db = mongoDb{DATABASE_URL} } else { // ... } err := api.Server(db) if err != nil { fmt.Println("Error:", err) }

Fiz um exemplo executável se ajudar: https://go.dev/play/p/RAYi89lY1oi

@cuducos eu ainda fico meio confuso com a integraçaão do golang com interfaces, entao eu por enquanto estou estudando mais essa minha deficiencia para retormar. enquanto isso tirei as linhas vazias.

vou retomar essa estrutura mais enxuta depois dos estudos.

cmd/cmd.go

cmd/transform.go

cmd/cmd.go

cuducos

E, finalmente tive tempo de revisar o mongodb.go — deixei alguns comentários, nada complicado, está bem bom!

Caso inglês seja uma barreira, deixe uns // TODO: translate q eu dou um tapa depois!

db/mongodb.go

db/mongo_test.go

.env.sample

cuducos · 2025-02-03T13:26:33Z

cmd/api.go

+
+		uri := os.Getenv("DATABASE_URL")
+		if strings.HasPrefix(uri, "mongodb://") {
+			mdb, _ := db.NewMongoDB(uri)
+			if err != nil {
+				return err
+			}
+
+			defer mdb.Close()
+			api.Serve(&mdb, port, nr)
+
+		} else if strings.HasPrefix(uri, "postgres://") || strings.HasPrefix(uri, "postgresql://") {
+			pg, err := db.NewPostgreSQL(u, postgresSchema, nr)
+			if err != nil {
+				return err
+			}
+			defer pg.Close()
+
+			api.Serve(&pg, port, nr)
+		} else {
+			return fmt.Errorf("unsupported database URI, must start with postgres:// or mongodb://")
+		}
+


Essa implementação precisa mudar usando a interface, para evitar repetição.

cuducos · 2025-02-03T13:27:27Z

cmd/cmd.go

+		if strings.HasPrefix(uri, "mongodb://") {
+			mdb, err := db.NewMongoDB(uri)
+			if err != nil {
+				return err
+			}
+			err = mdb.CreateCollection()
+			if err != nil {
+				return err
+			}
+			err = mdb.CreateIndexes()
+			if err != nil {
+				return err
+			}
+			defer mdb.Close()
 			return err
+		} else if strings.HasPrefix(uri, "postgres://") || strings.HasPrefix(uri, "postgresql://") {
+			pg, err := db.NewPostgreSQL(u, postgresSchema, nil)
+			if err != nil {
+				return err
+			}
+			defer pg.Close()
+			return pg.CreateTable()
+		} else {
+			return fmt.Errorf("A URL não contém 'mongodb' nem 'postgres'")


Essa implementação precisa mudar usando a interface, para evitar repetição.

cuducos · 2025-02-03T13:27:35Z

cmd/cmd.go

+		uri := os.Getenv("DATABASE_URL")
+		if strings.HasPrefix(uri, "mongodb://") {
+			mdb, err := db.NewMongoDB(uri)
+			if err != nil {
+				return err
+			}
+			err = mdb.DropCollection()
+			if err != nil {
+				return err
+			}
 			return err
+		} else if strings.HasPrefix(uri, "postgres://") || strings.HasPrefix(uri, "postgresql://") {
+			pg, err := db.NewPostgreSQL(u, postgresSchema, nil)
+			if err != nil {
+				return err
+			}
+			defer pg.Close()
+			return pg.DropTable()
+		} else {
+			return fmt.Errorf("A URL não contém 'mongodb' nem 'postgres'")


Essa implementação precisa mudar usando a interface, para evitar repetição.

cuducos · 2025-02-03T13:28:05Z

cmd/transform.go

-		if cleanUp {
-			if err := pg.DropTable(); err != nil {
+		uri := os.Getenv("DATABASE_URL")
+		if strings.HasPrefix(uri, "mongodb://") {


Essa implementação precisa mudar usando a interface, para evitar repetição.

db/mongodb.go

Co-authored-by: Eduardo Cuducos <[email protected]>

insert in mongo working fix interface testes mongodb consulta updated e o cnpj ja funcionando no mongo. restaurando o github url original fixes defer and recover debugs falta de conhecimento talvez? removido os debugs fixes ajustes solicitados fix aplicando mesmo padrao mais fixes alguns fixes fix PR traducao fixes exportei company, partenerData e cnae mudança no createIndex para o PostLoad e algumas traducoes retirado linhas vazias e nao ter necessidade da uri novamente refix translate and remove lines Update .env.sample Co-authored-by: Eduardo Cuducos <[email protected]> new fixes CreateCompanies golang mod tentativa de resolver conflitos tentando corrigir conflito variavel repetida fix again removing snappy novo go.mod e go.sum para o conflict Revert "novo go.mod e go.sum para o conflict" This reverts commit b0bc749. Update db/mongodb.go Co-authored-by: Eduardo Cuducos <[email protected]> Update db/mongodb.go Co-authored-by: Eduardo Cuducos <[email protected]> Update db/mongodb.go Co-authored-by: Eduardo Cuducos <[email protected]> Update db/mongodb.go Co-authored-by: Eduardo Cuducos <[email protected]> Update db/mongodb.go Co-authored-by: Eduardo Cuducos <[email protected]> Update db/mongodb.go Co-authored-by: Eduardo Cuducos <[email protected]> Update db/mongodb.go Co-authored-by: Eduardo Cuducos <[email protected]> Update db/mongodb.go Co-authored-by: Eduardo Cuducos <[email protected]> Update db/mongodb.go Co-authored-by: Eduardo Cuducos <[email protected]> Update db/mongodb.go Co-authored-by: Eduardo Cuducos <[email protected]> Update db/mongodb.go Co-authored-by: Eduardo Cuducos <[email protected]> Update db/mongodb.go Co-authored-by: Eduardo Cuducos <[email protected]> Update db/mongodb.go Co-authored-by: Eduardo Cuducos <[email protected]> Update db/mongodb.go Co-authored-by: Eduardo Cuducos <[email protected]> Update db/mongodb.go Co-authored-by: Eduardo Cuducos <[email protected]> Update db/mongodb.go Co-authored-by: Eduardo Cuducos <[email protected]> Update db/mongodb.go Co-authored-by: Eduardo Cuducos <[email protected]> Update db/mongodb.go Co-authored-by: Eduardo Cuducos <[email protected]> Update db/mongodb.go Co-authored-by: Eduardo Cuducos <[email protected]>

cmd/cmd.go

cmd/transform.go

db/mongodb.go

cuducos

O programa não compila por um erro de import cíclico, está na CI.

Localmente vc pode tentar com:

$ go test --race ./...
# github.com/cuducos/minha-receita/transform
package github.com/cuducos/minha-receita/transform
	imports github.com/cuducos/minha-receita/db
	imports github.com/cuducos/minha-receita/transform: import cycle not allowed in tes

Me parece q o múdulo transform depende de db e o módulo db depende do transform. Isso não pode. Lembro que sugeri num comentário:

Se quiser criar um módulo models com ele, pode ser um opção…

Não sei se é esse o caso, mas não consegui aprofundar ainda.

mbnunes added 10 commits January 22, 2025 16:59

first commit mongodb

f4e7b00

insert in mongo working

7efa454

fix interface

c4ea1a4

testes mongodb

6709df5

consulta updated e o cnpj ja funcionando no mongo.

a3f4a27

restaurando o github url original

157b65a

fixes

200e754

defer and recover debugs

d1172dd

falta de conhecimento talvez?

58c87e1

removido os debugs

41f99ed

cuducos changed the title ~~Mongodb Added~~ Adiciona possibilidade de uso com MongoDB Jan 29, 2025

cuducos reviewed Jan 29, 2025

View reviewed changes

mbnunes added 5 commits January 29, 2025 18:12

fixes

459350b

ajustes solicitados

106abd4

fix

befacea

aplicando mesmo padrao

2619c3a

mais fixes

d32d63b

cuducos reviewed Jan 30, 2025

View reviewed changes

mbnunes added 4 commits January 31, 2025 01:13

alguns fixes

1f67f18

fix PR

f999598

traducao

4f8fcb1

fixes

ea16a80

cuducos reviewed Feb 3, 2025

View reviewed changes

mbnunes added 2 commits February 3, 2025 10:50

exportei company, partenerData e cnae

d2f00d0

mudança no createIndex para o PostLoad e algumas traducoes

7cd37a7

cuducos reviewed Feb 3, 2025

View reviewed changes

db/mongodb.go Outdated Show resolved Hide resolved

db/mongodb.go Outdated Show resolved Hide resolved

db/mongodb.go Show resolved Hide resolved

db/mongodb.go Outdated Show resolved Hide resolved

mbnunes added 3 commits February 3, 2025 12:41

retirado linhas vazias e nao ter necessidade da uri novamente

77d8408

refix

1d79030

translate and remove lines

7e8116d

mbnunes and others added 20 commits February 6, 2025 19:56

Update db/mongodb.go

7dcde69

Co-authored-by: Eduardo Cuducos <[email protected]>

Update db/mongodb.go

5822940

Co-authored-by: Eduardo Cuducos <[email protected]>

Update db/mongodb.go

7120eca

Co-authored-by: Eduardo Cuducos <[email protected]>

Update db/mongodb.go

ecdfe97

Co-authored-by: Eduardo Cuducos <[email protected]>

Update db/mongodb.go

5a7fa61

Co-authored-by: Eduardo Cuducos <[email protected]>

Update db/mongodb.go

1ca1b60

Co-authored-by: Eduardo Cuducos <[email protected]>

Update db/mongodb.go

3c2888b

Co-authored-by: Eduardo Cuducos <[email protected]>

Update db/mongodb.go

5862d19

Co-authored-by: Eduardo Cuducos <[email protected]>

Update db/mongodb.go

f81cea5

Co-authored-by: Eduardo Cuducos <[email protected]>

Update db/mongodb.go

65e7aca

Co-authored-by: Eduardo Cuducos <[email protected]>

Update db/mongodb.go

36ddffe

Co-authored-by: Eduardo Cuducos <[email protected]>

Update db/mongodb.go

84dc685

Co-authored-by: Eduardo Cuducos <[email protected]>

Update db/mongodb.go

5a43234

Co-authored-by: Eduardo Cuducos <[email protected]>

Update db/mongodb.go

b00efca

Co-authored-by: Eduardo Cuducos <[email protected]>

Update db/mongodb.go

100ce65

Co-authored-by: Eduardo Cuducos <[email protected]>

Update db/mongodb.go

1a593b1

Co-authored-by: Eduardo Cuducos <[email protected]>

Update db/mongodb.go

c9695b0

Co-authored-by: Eduardo Cuducos <[email protected]>

Merge remote-tracking branch 'origin/mongodb' into mongodb

63abd6d

last fix

893efad

mbnunes requested a review from cuducos February 7, 2025 00:19

mbnunes and others added 5 commits February 7, 2025 14:40

adicionado comentario sobre o []interface{}

71ac1ff

tentando resolver a falta do snappy

dd6f812

sem snappy

c3d11de

ok nao mexo mais nisso :D

a3d7c3e

Merge branch 'main' into mongodb

814b538

cuducos requested changes Feb 7, 2025

View reviewed changes

cmd/cmd.go Outdated Show resolved Hide resolved

cmd/cmd.go Outdated Show resolved Hide resolved

cmd/transform.go Outdated Show resolved Hide resolved

db/mongodb.go Outdated Show resolved Hide resolved

ci ajustes

329d11c

cuducos requested changes Feb 7, 2025

View reviewed changes

Teste do CI

77f3f4b

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adiciona possibilidade de uso com MongoDB #278

Adiciona possibilidade de uso com MongoDB #278

mbnunes commented Jan 29, 2025 •

edited by cuducos

Loading

cuducos left a comment

cuducos left a comment

cuducos Jan 30, 2025

mbnunes Feb 3, 2025

cuducos left a comment

cuducos Feb 3, 2025

cuducos Feb 3, 2025

cuducos Feb 3, 2025

cuducos Feb 3, 2025

cuducos left a comment

Adiciona possibilidade de uso com MongoDB #278

Are you sure you want to change the base?

Adiciona possibilidade de uso com MongoDB #278

Conversation

mbnunes commented Jan 29, 2025 • edited by cuducos Loading

cuducos left a comment

Choose a reason for hiding this comment

cuducos left a comment

Choose a reason for hiding this comment

cuducos Jan 30, 2025

Choose a reason for hiding this comment

mbnunes Feb 3, 2025

Choose a reason for hiding this comment

cuducos left a comment

Choose a reason for hiding this comment

cuducos Feb 3, 2025

Choose a reason for hiding this comment

cuducos Feb 3, 2025

Choose a reason for hiding this comment

cuducos Feb 3, 2025

Choose a reason for hiding this comment

cuducos Feb 3, 2025

Choose a reason for hiding this comment

cuducos left a comment

Choose a reason for hiding this comment

mbnunes commented Jan 29, 2025 •

edited by cuducos

Loading