Big Data MLOps Platform

Data Science and Machine Learning Jupyter Notebooks

Tech Stack

shortthirdman/DataScience-Jupyter-Notebooks is built on the following main stack:

• Python – Languages
• Docker – Virtual Machine Platforms & Containers
• GitHub Actions – Continuous Integration
• Jupyter – Data Science Notebooks

Full tech stack here

Setup References

• Ready-to-run Docker images containing Jupyter applications - jupyter/docker-stacks

• Kaggle/docker-python - Kaggle Python Docker image

Dataset References

• Apple Stock Price

• Spotify Dataset 2023

• Online Retail Transactions

• Stock Market

• American Airlines Group Stock

• Amazon (AMZN) Historical Stock Price

Docker commands

 docker system prune --all --volumes --force

 docker build --no-cache -f Dockerfile --progress=auto --compress --rm -t shortthirdman-org/bigdata-mlops-platform:latest .

docker buildx build --progress=auto --compress --rm -t shortthirdman-org/bigdata-mlops-platform:latest .

 docker run -d -n mlops -p 8888:8888 --restart unless-stopped shortthirdman-org/bigdata-mlops-platform:latest

Local Setup

• Create a Python virtual environment and activate

  python -m venv dev

  .\dev\Scripts\activate

• Install the packages and dependencies as listed in requirements file

  pip install -r requirements.txt --no-cache-dir --disable-pip-version-check

• Start your development Jupyter Notebook or Jupyter Lab server

  jupyter lab --notebook-dir=.\notebooks --no-browser

  jupyter notebook

  jupyter_nbextensions_configurator
  

References

• TimeGPT: The First Foundation Model for Time Series Forecasting

• Staggering Returns with PyCryptoBot

• Trading Data Analysis

• Phenomenal Returns with PyCryptoBot

• Leveraging PyCryptoBot for Optimal Cryptocurrency Trading

• Forecasting Stock Using Deep Learning Along With Indicators | Medium

• Forecasting Stock Using Deep Learning Along With Indicators | OnePageCode@SubStack

• Advanced Stock Pattern Prediction using LSTM with the Attention Mechanism in TensorFlow: A step by step Guide with Apple Inc. (AAPL) Data

• Spark and Docker: Your Spark development cycle just got 10x faster!

• datamechanics/spark on Docker Hub

• Setting up a Spark standalone cluster on Docker in layman terms

• Apache Spark Standalone Cluster on Docker

• Visualizing Trading Signals in Python - Plot buy and sell trading signals in Python's graph

• Apache Hadoop and Apache Spark for Big Data Analysis

• Agent-Based Stock Trading: Design and Implementation

• Mastering K-Means Clustering

• Additive Decision Trees - An interpretable classification and regression model

• Interpretable kNN (ikNN) - An interpretable classifier

• Interpretable Outlier Detection: Frequent Patterns Outlier Factor (FPOF)

• Telco Customer Churn - Kaggle

• End-to-End Machine Learning Project: TelCo Churn Prediction

• Predicting Stock Prices with Monte Carlo Simulations

• Artificial Intelligence (AI) models for Trading - Exploring Random Forests from Machine Learning

• XGBoost for Stock Price Forecasting

• Model Interpretability Using Credit Card Fraud Data

• How Many Pokémon Fit?

• Gold Price Prediction Using Machine Learning

• Data Leakage in Preprocessing, Explained: A Visual Guide with Code Examples