archi@data

Data Architecture

Big Data, Streaming

Apache Kafka

• Kafka can be used for activity monitoring. The activity could belong to a website or physical sensors and devices. Producers can publish raw data from data sources that later can be used to find trends and pattern. From https://towardsdatascience.com/getting-started-with-apache-kafka-in-python-604b3250aa05
• Getting Started with Kafka & python: https://towardsdatascience.com/getting-started-with-apache-kafka-in-python-604b3250aa05
• Confluent's blog
  • Overview: https://www.confluent.io/blog/stream-data-platform-1/ and https://www.confluent.io/blog/stream-data-platform-2/
  • Kafka for Data Scientists: https://www.confluent.io/blog/putting-power-apache-kafka-hands-data-scientists/

QuasarDB

• https://quasardb.net/
  • .NET Client: https://github.com/bureau14/qdb-api-dotnet
  • Windows Administration: https://github.com/quasar/QuasarRAT
  • QuasarDB + Kafka:
    • https://doc.quasardb.net/master/api/kafka.html
    • https://github.com/bureau14/kafka-connect-qdb
    • https://jujucharms.com/u/bigdata-dev/kafka-quasardb/

Juju

• https://jujucharms.com/
  • https://jujucharms.com/big-data
  • https://jujucharms.com/how-it-works

Machining Data

• MTConnect vs OPC-UA: https://www.mmsonline.com/columns/the-difference-between-mtconnect-and-opc-ua

MTConnect

OPC-UA

MQTT