A command-line tool for launching Apache Spark clusters.

Scripts used to setup a Spark cluster on EC2

Automated Real-Time Indian Railway Twitter Complaint Administration System. It uses Apache Kafka, Spark, MySQL, PHP. The full project was deployed on AWS EC2 and RDS.

Recommendation System, Collaborative Filtering, Spark, Hive, Flask, Web Crawler, AWS EC2, AWS RDS

Image and VM management for Jenkins, Spark and Mesos clusters in EC2

Scripts used to setup a Spark cluster on EC2

Spark-cloud is a set of scripts for starting spark clusters on ec2

No description available

Scripts to deploy a GeoTrellis Spark cluster on EC2

Sbt plugin to submit Spark jobs to AWS EMR Spark Clusters

SBT plugin for spark-ec2

Data Pipeline examples using Oozie, Spark and Hive on Cloudera VM and AWS EC2 (branch aws-ec2)

FULL stack data science project (tech currently utilized: AWS/boto3/EMR/EC2/S3, Python, PySpark (Spark SQL and MLlib), and Flask/Flask RESTPlus)

No description available

Scrapped tweets using twitter API (for keyword ‘Netflix’) on an AWS EC2 instance, ingested data into S3 via kinesis firehose. Used Spark ML on databricks to build a pipeline for sentiment classification model and Athena & QuickSight to build a dashboard

Spark, EMR, EC2, Redshift, Glue

Setup procedure to work with jupyter notebook and pyspark on a EC2 AWS instance

Ansible playbook to setup apache spark and hdfs on AWS EC2

This project demonstrates skills in data engineering, specifically it contains an efficient ETL process utilizing AWS EC2, EMR and S3, Python and Spark and orchestrating the data pipeline with Airflow

Grad Course Work for ITCS-6190 Cloud Computing for Data Analysis. Stack Used : AWS , EC2 Clusters ,Spark,Spring Boot Applications

This project simulates a real-world enterprise data migration and modernization strategy. It extracts transactional data from a simulated "On-Premise" environment (hosted on AWS EC2), performs heavy distributed processing using a Hadoop/Spark cluster, and ultimately serves the data via a Cloud-Native, serverless architecture to optimize costs .

Spark and Python for Big Data with PySpark (SparkML, DataFrames) Udemy course projects

No description available

Spark ML, Spark SQL, Spark DataFrame, AWS EMR, AWS S3, AWS EC2, ML Classification

MY work around AWS (includes NGINX,EC2,S3,Autoscaling,RDS,BeanStalk,Hadoop,Spark)

Overcome mislabeling errors in genomics training sets by utilizing machine learning on AWS EC2 and Apache Spark.

Build a recommendation system for Twitter hashtags using Neo4j graph database running on Spark GraphX on an EC2 cluster on AWS.

Implemented spatial hotspot analysis on the NYC Yellow Cab taxi trip records using spark cluster setup on the AWS EC2 Instances. The aim was to analyse huge dataset using distributed cluster-computing framework like Apache Spark and Apache Sedona.

👷🌇 Set up and build a big data processing pipeline with Apache Spark, 📦 AWS services (S3, EMR, EC2, IAM, VPC, Redshift) Terraform to setup the infrastructure and Integration Airflow to automate workflows🥊

A project about building a stacked model with tuning the hyperparameters with grid search and hyperopt and used PySpark to test the performance of model in Spark Clusters in AWS EC2 and ROSE to balance the target variable