Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release/v1.0.0 #10

Open
wants to merge 97 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
97 commits
Select commit Hold shift + click to select a range
6bf7411
Created file to run project stacks with docker containers
pedrohfocus Feb 10, 2022
3c139d2
Create global environment vars for compose file
pedrohfocus Feb 10, 2022
d1590b2
Created Dockerfile
pedrohfocus Feb 10, 2022
e749806
Added api stack dependences
pedrohfocus Feb 10, 2022
1a2da65
Created typescript config file
pedrohfocus Feb 10, 2022
d4abacc
Created .dockerignore with node_modules rule
pedrohfocus Feb 10, 2022
54b0558
Created server file
pedrohfocus Feb 10, 2022
ebf89c3
Created api stack validators layer
pedrohfocus Feb 10, 2022
771047a
Created api stack infra layer
pedrohfocus Feb 10, 2022
979bff0
Created api stack entities layer
pedrohfocus Feb 10, 2022
4c9e411
Change sequelize schema struct
pedrohfocus Feb 11, 2022
503b03a
Updated sequelize model struct, Field types, Created table aggregation
pedrohfocus Feb 11, 2022
9756c72
Updated sequelize schema struct, changed code field to unique and sho…
pedrohfocus Feb 11, 2022
c3098ec
Added promise return type to repository interface
pedrohfocus Feb 11, 2022
98629df
Added options support to repository methods
pedrohfocus Feb 11, 2022
2d703ec
Created api stack routes
pedrohfocus Feb 11, 2022
d03c586
Created api stack stores domain
pedrohfocus Feb 11, 2022
da75482
Created api stack transactions domain
pedrohfocus Feb 11, 2022
7ef783d
Created api stack transactions types domain
pedrohfocus Feb 11, 2022
bbfc205
Created domain layer type file to add domain global types and interfaces
pedrohfocus Feb 11, 2022
e98d4f6
package-lock.json cache
pedrohfocus Feb 11, 2022
cf1977e
Intialized parser module
pedrohfocus Feb 11, 2022
061439e
Added parser, kafka and kafdrop services
pedrohfocus Feb 11, 2022
86467a0
Updated environment variables
pedrohfocus Feb 11, 2022
abb5e45
Enable express urlencoded
pedrohfocus Feb 11, 2022
3f64c30
Added multer package
pedrohfocus Feb 11, 2022
98e7746
Created upload router
pedrohfocus Feb 11, 2022
c9c30d2
Created services layer with kafka producer
pedrohfocus Feb 11, 2022
6c73608
Created upload to parse domain
pedrohfocus Feb 11, 2022
8e28194
.gitignore updated
pedrohfocus Feb 11, 2022
8944722
Created parser stack providers layer
pedrohfocus Feb 14, 2022
dd0f11f
Created parser stack entities
pedrohfocus Feb 14, 2022
e2fa391
Created infra layer to management of repositories layer
pedrohfocus Feb 14, 2022
6986ba9
Created parser stack helpers
pedrohfocus Feb 14, 2022
d51598d
Created parser stack .gitignore rules
pedrohfocus Feb 14, 2022
d0868dc
Added new packages to project requirements
pedrohfocus Feb 14, 2022
e2563a0
Integrated parser stack with server api and kafka cluster
pedrohfocus Feb 14, 2022
94095d7
Updated services to docker compose
pedrohfocus Feb 14, 2022
bf57dd0
Created seeds service to populate transactions types consumed by pars…
pedrohfocus Feb 14, 2022
1af99f0
Enable to sent custom id in request body
pedrohfocus Feb 14, 2022
a872a26
Improviments in repository
pedrohfocus Feb 14, 2022
67cf65c
Created new schema and entity to manage parser processing status
pedrohfocus Feb 15, 2022
3383171
Integrate parse entity when receive a file from upload form
pedrohfocus Feb 15, 2022
a800ca9
Minor ajust. Removed console log
pedrohfocus Feb 15, 2022
e98b07c
Added uuid package to create custom uuids
pedrohfocus Feb 15, 2022
f348c38
Removed python cache folders
pedrohfocus Feb 15, 2022
b336a89
Added frontend stack to docker compose
pedrohfocus Feb 15, 2022
b09edff
Created Dockerfile frontend stack
pedrohfocus Feb 15, 2022
095a6b3
Added project ignores
pedrohfocus Feb 15, 2022
502f69e
Added project package file
pedrohfocus Feb 15, 2022
b6eee77
Added frontend stack project files
pedrohfocus Feb 15, 2022
a963392
Created App Preloader
pedrohfocus Feb 15, 2022
2a2bb3a
Create frontend stack provider layer
pedrohfocus Feb 15, 2022
e3180a3
Created routes components to private and public
pedrohfocus Feb 15, 2022
5715324
Created frontend stack utils layer
pedrohfocus Feb 15, 2022
236c5ef
Updated app global styles with design system params
pedrohfocus Feb 15, 2022
d48b3ee
Added frontend stack redux store provider and defined global styles a…
pedrohfocus Feb 15, 2022
3f457b7
Added auth page styled components ds
pedrohfocus Feb 15, 2022
9f817e9
Created auth component UI and connected with redux provider
pedrohfocus Feb 15, 2022
ccfd90f
Created frontend stack redux store
pedrohfocus Feb 15, 2022
bbe1263
Added redux dependences and api server proxy
pedrohfocus Feb 15, 2022
efafaa9
Updated frontend stack app component with routes switch
pedrohfocus Feb 15, 2022
c3ad4ee
Added authenticate router to express router map
pedrohfocus Feb 15, 2022
0dbf02b
Defined secret value on env vars
pedrohfocus Feb 15, 2022
0d5ab08
Added jsonwebtoken dependences
pedrohfocus Feb 15, 2022
64b83fe
Created authenticate domain handle
pedrohfocus Feb 15, 2022
3bec8da
Created App module
pedrohfocus Feb 15, 2022
e0295fc
Added page module to main switch
pedrohfocus Feb 15, 2022
2cd53bc
Created user redux module
pedrohfocus Feb 15, 2022
5a8277c
Removed unused imports
pedrohfocus Feb 15, 2022
0bb7f87
Created app module pages layer
pedrohfocus Feb 15, 2022
1be0a73
Created stores redux module
pedrohfocus Feb 15, 2022
64a0f20
Added stores module to redux roots
pedrohfocus Feb 15, 2022
862130f
chore ajust
pedrohfocus Feb 15, 2022
7a2da06
Added stores page to app module switch rotues
pedrohfocus Feb 15, 2022
0294275
Moved stores to app module pages
pedrohfocus Feb 15, 2022
00e002f
Remove unused component from stores page component
pedrohfocus Feb 15, 2022
bc8340d
Created frontend stack portfolio redux module
pedrohfocus Feb 16, 2022
cd587ed
Created portfolio page
pedrohfocus Feb 16, 2022
9e931f6
Created portfolio module to roots
pedrohfocus Feb 16, 2022
bd152bd
Added new packages
pedrohfocus Feb 16, 2022
0ca8e26
Added portfolio pages in app module
pedrohfocus Feb 16, 2022
a036d05
Integrate Parser Status in all application layers
pedrohfocus Feb 16, 2022
4baf492
Refactored seeds stack
pedrohfocus Feb 16, 2022
9fd7e49
Project documentation
pedrohfocus Feb 16, 2022
faa79f0
Removed cached
pedrohfocus Feb 16, 2022
99eee46
Seeds gitignore
pedrohfocus Feb 16, 2022
8e7eb25
Finished readme doc
pedrohfocus Feb 16, 2022
125e65d
Created frontend stack upload and delete redux modules
pedrohfocus Feb 16, 2022
4224d39
Added swal alert package
pedrohfocus Feb 16, 2022
d7c16da
Created global button style
pedrohfocus Feb 16, 2022
2690fbc
Integrated upload function
pedrohfocus Feb 16, 2022
5e0c045
Created delete method on API service provider
pedrohfocus Feb 16, 2022
ce97c65
Added delete resource on api routes
pedrohfocus Feb 16, 2022
c7229a3
Created delete parser resource
pedrohfocus Feb 16, 2022
c282daf
Added animated usage presentation
pedrohfocus Feb 16, 2022
c999b40
Added gif presentation
pedrohfocus Feb 16, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions .env
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
DBNAME=desafiodev
DBUSER=desafiodev
DBPASSWORD=desafiodev
DBHOST=postgresdb

ZOOKEEPER_CLIENT_PORT=2181
ZOOKEEPER_TICK_TIME=2000

KAFKA_TOPIC_NAME=parser_transactions
KAFKA_INTERNAL_HOST=kafka
KAFKA_INTERNAL_PORT=29092
KAFKA_EXTERNAL_HOST=localhost
KAFKA_EXTERNAL_PORT=9092
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
**/node_modules/*
**/venv/*
**/.idea/*
**/.vscode/*
201 changes: 116 additions & 85 deletions README.md
100755 → 100644
Original file line number Diff line number Diff line change
@@ -1,85 +1,116 @@
# Desafio programação - para vaga desenvolvedor

Por favor leiam este documento do começo ao fim, com muita atenção.
O intuito deste teste é avaliar seus conhecimentos técnicos em programação.
O teste consiste em parsear [este arquivo de texto(CNAB)](https://github.com/ByCodersTec/desafio-ruby-on-rails/blob/master/CNAB.txt) e salvar suas informações(transações financeiras) em uma base de dados a critério do candidato.
Este desafio deve ser feito por você em sua casa. Gaste o tempo que você quiser, porém normalmente você não deve precisar de mais do que algumas horas.

# Instruções de entrega do desafio

1. Primeiro, faça um fork deste projeto para sua conta no Github (crie uma se você não possuir).
2. Em seguida, implemente o projeto tal qual descrito abaixo, em seu clone local.
3. Por fim, envie via email o projeto ou o fork/link do projeto para seu contato Bycoders_ com cópia para [email protected].

# Descrição do projeto

Você recebeu um arquivo CNAB com os dados das movimentações finanaceira de várias lojas.
Precisamos criar uma maneira para que estes dados sejam importados para um banco de dados.

Sua tarefa é criar uma interface web que aceite upload do [arquivo CNAB](https://github.com/ByCodersTec/desafio-ruby-on-rails/blob/master/CNAB.txt), normalize os dados e armazene-os em um banco de dados relacional e exiba essas informações em tela.

**Sua aplicação web DEVE:**

1. Ter uma tela (via um formulário) para fazer o upload do arquivo(pontos extras se não usar um popular CSS Framework )
2. Interpretar ("parsear") o arquivo recebido, normalizar os dados, e salvar corretamente a informação em um banco de dados relacional, **se atente as documentações** que estão logo abaixo.
3. Exibir uma lista das operações importadas por lojas, e nesta lista deve conter um totalizador do saldo em conta
4. Ser escrita na sua linguagem de programação de preferência
5. Ser simples de configurar e rodar, funcionando em ambiente compatível com Unix (Linux ou Mac OS X). Ela deve utilizar apenas linguagens e bibliotecas livres ou gratuitas.
6. Git com commits atomicos e bem descritos
7. PostgreSQL, MySQL ou SQL Server
8. Ter testes automatizados
9. Docker compose (Pontos extras se utilizar)
10. Readme file descrevendo bem o projeto e seu setup
11. Incluir informação descrevendo como consumir o endpoint da API

**Sua aplicação web não precisa:**

1. Lidar com autenticação ou autorização (pontos extras se ela fizer, mais pontos extras se a autenticação for feita via OAuth).
2. Ser escrita usando algum framework específico (mas não há nada errado em usá-los também, use o que achar melhor).
3. Documentação da api.(Será um diferencial e pontos extras se fizer)

# Documentação do CNAB

| Descrição do campo | Inicio | Fim | Tamanho | Comentário
| ------------- | ------------- | -----| ---- | ------
| Tipo | 1 | 1 | 1 | Tipo da transação
| Data | 2 | 9 | 8 | Data da ocorrência
| Valor | 10 | 19 | 10 | Valor da movimentação. *Obs.* O valor encontrado no arquivo precisa ser divido por cem(valor / 100.00) para normalizá-lo.
| CPF | 20 | 30 | 11 | CPF do beneficiário
| Cartão | 31 | 42 | 12 | Cartão utilizado na transação
| Hora | 43 | 48 | 6 | Hora da ocorrência atendendo ao fuso de UTC-3
| Dono da loja | 49 | 62 | 14 | Nome do representante da loja
| Nome loja | 63 | 81 | 19 | Nome da loja

# Documentação sobre os tipos das transações

| Tipo | Descrição | Natureza | Sinal |
| ---- | -------- | --------- | ----- |
| 1 | Débito | Entrada | + |
| 2 | Boleto | Saída | - |
| 3 | Financiamento | Saída | - |
| 4 | Crédito | Entrada | + |
| 5 | Recebimento Empréstimo | Entrada | + |
| 6 | Vendas | Entrada | + |
| 7 | Recebimento TED | Entrada | + |
| 8 | Recebimento DOC | Entrada | + |
| 9 | Aluguel | Saída | - |

# Avaliação

Seu projeto será avaliado de acordo com os seguintes critérios.

1. Sua aplicação preenche os requerimentos básicos?
2. Você documentou a maneira de configurar o ambiente e rodar sua aplicação?
3. Você seguiu as instruções de envio do desafio?
4. Qualidade e cobertura dos testes unitários.

Adicionalmente, tentaremos verificar a sua familiarização com as bibliotecas padrões (standard libs), bem como sua experiência com programação orientada a objetos a partir da estrutura de seu projeto.

# Referência

Este desafio foi baseado neste outro desafio: https://github.com/lschallenges/data-engineering

---

Boa sorte!
<p align="center">
<a href="" rel="noopener">
<img src="./src/preview.png" alt="Repo preview">
</a>
</p>

<h3 align="center">MyTransactions</h3>

<div align="center">

[![Bitbucket open issues](https://img.shields.io/bitbucket/issues/preduus/desafio-dev?style=for-the-badge)](https://github.com/preduus/desafio-dev/issues)
[![GitHub forks](https://img.shields.io/github/forks/preduus/desafio-dev?style=for-the-badge)]()
[![GitHub stars](https://img.shields.io/github/stars/preduus/desafio-dev?style=for-the-badge)]()
[![GitHub forks](https://img.shields.io/github/license/preduus/desafio-dev?style=for-the-badge)](/LICENSE)

</div>

---


## 📝 Table of Contents

- [About](#about)
- [Installing](#installing)
- [Usage](#usage)
- [Built Using](#built_using)
- [License](#license)

## 🧐 About <a name="about"></a>

The idea of ​​the project is to process information related to financial transactions and store them in a database.
A simple approach was used, with some concepts of asynchronous processes in an ecosystem in docker and docker-compose, the communication is carried out through the internal docker network, simulating a private production environment.
Kafka was the library used to communicate between the services developed, with the aim of guaranteeing security in the processing of information and thinking about the UX.
## Prerequisites

What things you need to install the software and how to install them.

* [Docker](https://docs.docker.com/get-docker/)
* [Docker Compose](https://docs.docker.com/compose/install/)


## Installation

We chose to use [docker](https://docs.docker.com/get-docker/) and [docker compose](https://docs.docker.com/compose/install/) to develop this application. If you haven't installed it, you can find the installation steps [here](https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-on-ubuntu-20-04-pt).

## 🎈 Usage <a name="usage"></a>
<img src="./src/using.gif" />

**To see API Documentation, [click here](https://documenter.getpostman.com/view/6161616/UVkiRcxa).**

You just go up the stack and use it. Run:
```bash
$ docker-compose up -d

Starting desafio-dev_postgresdb_1 ... done
Starting desafio-dev_zookeeper_1 ... done
Starting desafio-dev_kafka_1 ... done
Starting desafiodev-api ... done
Starting desafio-dev_kafdrop_1 ... done
Starting desafiodev-parser ... done
Starting desafiodev-seeds ... done
Starting desafiodev-frontend ... done
Attaching to desafio-dev_zookeeper_1, desafio-dev_postgresdb_1, desafio-dev_kafka_1, desafio-dev_kafdrop_1, desafiodev-api, desafiodev-parser, desafiodev-seeds, desafiodev-frontend
```

### Services available:
* Frontend Microservice: http://localhost:8000
* Server API Microservice: http://localhost:3100
* PostgreSQL Service: http://localhost:5433
* Kafka Service: http://localhost:9092
* Kafdrop Service: http://localhost:19000

#### Fake login
* Username: **[email protected]**
* Password: **desafiodev**

The project structure is simple. There are 4 microservices and some services available in the environment.

#### Microservices:
* [Server API](./backend/api)
* [Parser Service](./backend/parser)
* [Database Seeds](./backend/seeds)
* [Frontend](./frontend)

#### Services:
* [Apache Kafka](https://kafka.apache.org/)
* [Zookeeper](https://zookeeper.apache.org/)
* [Kafdrop](https://github.com/obsidiandynamics/kafdrop)
* [PostgreSQL](https://www.postgresql.org/)


**If you want to test it, just import the [API collection](./collection.json) into Postman.**

You can change params in [**compose.yml**](compose.yml) file.


## ⛏️ Built Using <a name="built_using"></a>

- [Docker](https://www.docker.com/)
- [Node.js](https://nodejs.org/)
- [Python 3.8](https://www.python.org/downloads/release/python-380/)
- [Redux](https://redux.js.org/)
- [React Redux](https://react-redux.js.org/)
- [Redux Saga](https://redux-saga.js.org/)
- [Vercel SWR](https://swr.vercel.app/)
- [Express](https://expressjs.com/)
- [Joi](https://joi.dev/api/)
- [PostgreSQL](https://www.postgresql.org/)
- [Sequelize ORM](https://sequelize.org/)
- [Apache Kafka](https://kafka.apache.org/)
- [Zookeeper](https://zookeeper.apache.org/)
- [Kafdrop](https://github.com/obsidiandynamics/kafdrop)

## 📝 License <a name="license"></a>

Distributed under the MIT License. See LICENSE for more information.
1 change: 1 addition & 0 deletions backend/api/.dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
/node_modules/*
8 changes: 8 additions & 0 deletions backend/api/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
FROM node:16

WORKDIR /usr/src/api

COPY . .
RUN npm install

CMD ["npm", "start"]
Loading