MIT Introduction to Deep Learning (6.S191) Instructors: Alexander Amini and Ava Soleimany Course Information Summary Prerequisites Schedule Lectures Labs, Final Projects, Grading, and Prizes Software labs Gather.Town lab + Office Hour sessions Final project Paper Review Project Proposal Presentation Project Proposal Grading Rubric Past Project Proposal Ideas Awards + Categories Important Links and Emails Course Information Summary MIT's introductory course on deep learning methods with applications to computer vision, natural language processing, biology, and more! Students will gain foundational knowledge of deep learning algorithms and get practical experience in building neural networks in TensorFlow. Course concludes with a project proposal competition with feedback from staff and a panel of industry sponsors. Prerequisites We expect basic knowledge of calculus (e.g., taking derivatives), linear algebra (e.g., matrix multiplication), and probability (e.g., Bayes theorem) -- we'll try to explain everything else along the way! Experience in Python is helpful but not necessary. This class is taught during MIT's IAP term by current MIT PhD researchers. Listeners are welcome! Schedule Monday Jan 18, 2021 Lecture: Introduction to Deep Learning and NNs Lab: Lab 1A Tensorflow and building NNs from scratch Tuesday Jan 19, 2021 Lecture: Deep Sequence Modelling Lab: Lab 1B Music Generation using RNNs Wednesday Jan 20, 2021 Lecture: Deep Computer Vision Lab: Lab 2A Image classification and detection Thursday Jan 21, 2021 Lecture: Deep Generative Modelling Lab: Lab 2B Debiasing facial recognition systems Friday Jan 22, 2021 Lecture: Deep Reinforcement Learning Lab: Lab 3 pixel-to-control planning Monday Jan 25, 2021 Lecture: Limitations and New Frontiers Lab: Lab 3 continued Tuesday Jan 26, 2021 Lecture (part 1): Evidential Deep Learning Lecture (part 2): Bias and Fairness Lab: Work on final assignments Lab competition entries due at 11:59pm ET on Canvas! Lab 1, Lab 2, and Lab 3 Wednesday Jan 27, 2021 Lecture (part 1): Nigel Duffy, Ernst & Young Lecture (part 2): Kate Saenko, Boston University and MIT-IBM Watson AI Lab Lab: Work on final assignments Assignments due: Sign up for Final Project Competition Thursday Jan 28, 2021 Lecture (part 1): Sanja Fidler, U. Toronto, Vector Institute, and NVIDIA Lecture (part 2): Katherine Chou, Google Lab: Work on final assignments Assignments due: 1 page paper review (if applicable) Friday Jan 29, 2021 Lecture: Student project pitch competition Lab: Awards ceremony and prize giveaway Assignments due: Project proposals (if applicable) Lectures Lectures will be held starting at 1:00pm ET from Jan 18 - Jan 29 2021, Monday through Friday, virtually through Zoom. Current MIT students, faculty, postdocs, researchers, staff, etc. will be able to access the lectures during this two week period, synchronously or asynchronously, via the MIT Canvas course webpage (MIT internal only). Lecture recordings will be uploaded to the Canvas as soon as possible; students are not required to attend any lectures synchronously. Please see the Canvas for details on Zoom links. The public edition of the course will only be made available after completion of the MIT course. Labs, Final Projects, Grading, and Prizes Course will be graded during MIT IAP for 6 units under P/D/F grading. Receiving a passing grade requires completion of each software lab project (through honor code, with submission required to enter lab competitions), a final project proposal/presentation or written review of a deep learning paper (submission required), and attendance/lecture viewing (through honor code). Submission of a written report or presentation of a project proposal will ensure a passing grade. MIT students will be eligible for prizes and awards as part of the class competitions. There will be two parts to the competitions: (1) software labs and (2) final projects. More information is provided below. Winners will be announced on the last day of class, with thousands of dollars of prizes being given away! Software labs There are three TensorFlow software lab exercises for the course, designed as iPython notebooks hosted in Google Colab. Software labs can be found on GitHub: https://github.com/aamini/introtodeeplearning. These are self-paced exercises and are designed to help you gain practical experience implementing neural networks in TensorFlow. For registered MIT students, submission of lab materials is not necessary to get credit for the course or to pass the course. At the end of each software lab there will be task-associated materials to submit (along with instructions) for entry into the competitions, open to MIT students and affiliates during the IAP offering. This includes MIT students/affiliates who are taking the class as listeners -- you are eligible! These instructions are provided at the end of each of the labs. Completing these tasks and submitting your materials to Canvas will enter you into a per-lab competition. MIT students and affiliates will be eligible for prizes during the IAP offering; at the end of the course, prize-winners will be awarded with their prizes. All competition submissions are due on January 26 at 11:59pm ET to Canvas. For the software lab competitions, submissions will be judged on the basis of the following criteria: Strength and quality of final results (lab dependent) Soundness of implementation and approach Thoroughness and quality of provided descriptions and figures Gather.Town lab + Office Hour sessions After each day’s lecture, there will be open Office Hours in the class GatherTown, up until 3pm ET. An MIT email is required to log in and join the GatherTown. During these sessions, there will not be a walk through or dictation of the labs; the labs are designed to be self-paced and to be worked on on your own time. The GatherTown sessions will be hosted by course staff and are held so you can: Ask questions on course lectures, labs, logistics, project, or anything else; Work on the labs in the presence of classmates/TAs/instructors; Meet classmates to find groups for the final project; Group work time for the final project; Bring the class community together. Final project To satisfy the final project requirement for this course, students will have two options: (1) write a 1 page paper review (single-spaced) on a recent deep learning paper of your choice or (2) participate and present in the project proposal pitch competition. The 1 page paper review option is straightforward, we propose some papers within this document to help you get started, and you can satisfy a passing grade with this option -- you will not be eligible for the grand prizes. On the other hand, participation in the project proposal pitch competition will equivalently satisfy your course requirements but additionally make you eligible for the grand prizes. See the section below for more details and requirements for each of these options. Paper Review Students may satisfy the final project requirement by reading and reviewing a recent deep learning paper of their choosing. In the written review, students should provide both: 1) a description of the problem, technical approach, and results of the paper; 2) critical analysis and exposition of the limitations of the work and opportunities for future work. Reviews should be submitted on Canvas by Thursday Jan 28, 2021, 11:59:59pm Eastern Time (ET). Just a few paper options to consider... https://papers.nips.cc/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf https://papers.nips.cc/paper/2018/file/69386f6bb1dfed68692a24c8686939b9-Paper.pdf https://papers.nips.cc/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf https://science.sciencemag.org/content/362/6419/1140 https://papers.nips.cc/paper/2018/file/0e64a7b00c83e3d22ce6b3acf2c582b6-Paper.pdf https://arxiv.org/pdf/1906.11829.pdf https://www.nature.com/articles/s42256-020-00237-3 https://pubmed.ncbi.nlm.nih.gov/32084340/ Project Proposal Presentation Keyword: proposal This is a 2 week course so we do not require results or working implementations! However, to win the top prizes, nice, clear results and implementations will demonstrate feasibility of your proposal which is something we look for! Logistics -- please read! You must sign up to present before 11:59:59pm Eastern Time (ET) on Wednesday Jan 27, 2021 Slides must be in a Google Slide before 11:59:59pm Eastern Time (ET) on Thursday Jan 28, 2021 Project groups can be between 1 and 5 people Listeners welcome To be eligible for a prize you must have at least 1 registered MIT student in your group Each participant will only be allowed to be in one group and present one project pitch Synchronous attendance on 1/29/21 is required to make the project pitch! 3 min presentation on your idea (we will be very strict with the time limits) Prizes! (see below) Sign up to Present here: by 11:59pm ET on Wednesday Jan 27 Once you sign up, make your slide in the following Google Slides; submit by midnight on Thursday Jan 28. Please specify the project group # on your slides!!! Things to Consider This doesn’t have to be a new deep learning method. It can just be an interesting application that you apply some existing deep learning method to. What problem are you solving? Are there use cases/applications? Why do you think deep learning methods might be suited to this task? How have people done it before? Is it a new task? If so, what are similar tasks that people have worked on? In what aspects have they succeeded or failed? What is your method of solving this problem? What type of model + architecture would you use? Why? What is the data for this task? Do you need to make a dataset or is there one publicly available? What are the characteristics of the data? Is it sparse, messy, imbalanced? How would you deal with that? Project Proposal Grading Rubric Project proposals will be evaluated by a panel of judges on the basis of the following three criteria: 1) novelty and impact; 2) technical soundness, feasibility, and organization, including quality of any presented results; 3) clarity and presentation. Each judge will award a score from 1 (lowest) to 5 (highest) for each of the criteria; the average score from each judge across these criteria will then be averaged with that of the other judges to provide the final score. The proposals with the highest final scores will be selected for prizes. Here are the guidelines for the criteria: Novelty and impact: encompasses the potential impact of the project idea, its novelty with respect to existing approaches. Why does the proposed work matter? What problem(s) does it solve? Why are these problems important? Technical soundness, feasibility, and organization: encompasses all technical aspects of the proposal. Do the proposed methodology and architecture make sense? Is the architecture the best suited for the proposed problem? Is deep learning the best approach for the problem? How realistic is it to implement the idea? Was there any implementation of the method? If results and data are presented, we will evaluate the strength of the results/data. Clarity and presentation: encompasses the delivery and quality of the presentation itself. Is the talk well organized? Are the slides aesthetically compelling? Is there a clear, well-delivered narrative? Are the problem and proposed method clearly presented? Past Project Proposal Ideas Recipe Generation with RNNs Can we compress videos with CNN + RNN? Music Generation with RNNs Style Transfer Applied to X GAN’s on a new modality Summarizing text/news articles Combining news articles about similar events Code or spec generation Multimodal speech → handwriting Generate handwriting based on keywords (i.e. cursive, slanted, neat) Predicting stock market trends Show language learners articles or videos at their level Transfer of writing style Chemical Synthesis with Recurrent Neural networks Transfer learning to learn something in a domain for which it’s hard or risky to gather data or do training RNNs to model some type of time series data Computer vision to coach sports players Computer vision system for safety brakes or warnings Use IBM Watson API to get the sentiment of your Facebook newsfeed Deep learning webcam to give wifi-access to friends or improve video chat in some way Domain-specific chatbot to help you perform a specific task Detect whether a signature is fraudulent Awards + Categories Final Project Awards: 1x NVIDIA RTX 3080 4x Google Home Max 3x Display Monitors Software Lab Awards: Bose headphones (Lab 1) Display monitor (Lab 2) Bebop drone (Lab 3) Important Links and Emails Course website: http://introtodeeplearning.com Course staff: introtodeeplearning-staff@mit.edu Piazza forum (MIT only): https://piazza.com/mit/spring2021/6s191 Canvas (MIT only): https://canvas.mit.edu/courses/8291 Software lab repository: https://github.com/aamini/introtodeeplearning Lab/office hour sessions (MIT only): https://gather.town/app/56toTnlBrsKCyFgj/MITDeepLearning

Implementation of Logistic Regression, MLP, CNN, RNN & LSTM from scratch in python. Training of deep learning models for image classification, object detection, and sequence processing (including transformers implementation) in TensorFlow.

Classify MNIST image dataset into 10 classes. Build an image classifier with Recurrent Neural Network (RNN: LSTM) on Tensorflow.

Jupyter Notebooks with home-made FCNNs, RNNs, CNNs, GAMs models for Airbnb price rental prediction, galaxies classification, gravitational waves classification, named entity prediction, images classification, etc.

Paper implementation

斯坦福李飞飞深度学习课程的课后作业，有3个部分Assignment #1: Image Classification, kNN, SVM, Softmax, Neural NetworkAssignment #2: Fully-Connected Nets, Batch Normalization, Dropout, Convolutional NetsAssignment #3: Image Captioning with Vanilla RNNs, Image Captioning with LSTMs, Network Visualization, Style Transfer, Generative Adversarial Networks官方资源（讲义、作业等）地址：http://cs231n.github.io/网易课程地址：http://study.163.com/course/courseMain.htm?courseId=1003223001

The Optical Character Recognition (OCR) system consists of a comprehensive neural network built using Python and TensorFlow that was trained on over 115,000 wordimages from the IAM On-Line Handwriting Database (IAM-OnDB). The neural network consists of 5 Convolutional Neural Network (CNN) layers, 2 Recurrent Neural Network (RNN) Layers, and a final Connectionist Temporal Classification (CTC) layer. As the input image is fed into the CNN layers, a non-linear ReLU function is applied to extract relevant features from the image. The ReLU function is preferred due to the lower likelihood of a vanishing gradient (which arises when network parameters and hyperparameters are not properly set) relative to a sigmoid function. In the case of the RNN layers, the Long Short-Term Memory (LSTM) implementation is used due to its ability to propagate information through long distances. The CTC is given the RNN output matrix and the ground truth text to compute the loss value and the mean of the loss values of the batch elements is used to train the OCR system. This means is fed into an RMSProp optimizer which is focused on minimizing the loss, and it does so in a very robust manner. For inference, the CTC layer decodes the RNN output matrix into the final text. The OCR system reports an accuracy rate of 95.7% for the IAM Test Dataset, but this accuracy falls to 89.4% for unseen handwritten doctors’ prescriptions.

The following model uses hybrid CNN- RNN model for classification of each pixel to its corresponding classes. Further the code is developed to classify pixels in accordance with soft as well as hard classification techniques.

CNN-RNN: A Unified Framework for Multi-Label Image Classification

Using Deep ConvNets, RNNs, LSTMs, GANs and more from Image Classification to Face Generation.

image classification, style transfer, deep dream, rigid icp, char-rnn, backprop, variational autoencoder, ...

The course is offered by KTH in Autumn semester and and focuses on Medical image segmentation using CNN and hands-on section with TensorFlow, ,medical image classification using CNN and hands-on section with TensorFlow, medical image analysis using RNN and hands-on section with TensorFlow

Image classification (RNN/LSTM/GRU/Bidirectional-LSTM).

This repository contain the In-Class projects I did in the machine learning course. The project topics include PM2.5 prediction, the annual salary prediction, image sentiment classification, content classification and movie recommend system. The ML technology used in these projects are linear regression, logistic regression, CNN, RNN and Metrix Factorization.

Here I used LSTM and RNN for image classification. The dataset used is Mnist.

Word / sentence generation with trainable RNN and encoder-decoders; applications in image caption generation and classification tasks

Projects such as Image classification using CNN, Object detection using YOLO algorithm, face recognition and verification, neural style transfer using RNN algorithm are included.

This project focuses on land use and land cover classification using Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs). The classification task aims to predict the category of land based on satellite or aerial images.

A Convolutional neural network (CNN) is a neural network that has one or more convolutional layers and are used mainly for image processing, classification, segmentation and also for other auto correlated data.CNNs have been used for understanding in Natural Language Processing (NLP) and speech recognition, although often for NLP Recurrent Neural Nets (RNNs) are used.

Module 2 – Distributed Deep Learning: Introduction to the theory and implementation of distributed deep learning: Classification and regression using generalized linear models, including different learning, regularization, and hyperparameters tuning techniques. The feedforward deep network as a fundamental network, and the advanced techniques to overcome its main challenges, such as overfitting, vanishing/exploding gradient, and training speed. Various deep neural networks for various kinds of data. For example, the CNN for scaling up neural networks to process large images, RNN to scale up deep neural models to long temporal sequences, and autoencoder and GANs. In this course module, we aim to ensure that all students understand the basic concepts and tools in distributed deep learning.

图像二分类

Exploration of RNNs and GNNs in image classification

The following model uses hybrid CNN- RNN model for classification of each pixel to its corresponding classes. Further the code is developed to classify pixels in accordance with soft as well as hard classification techniques.

This project focuses on land use and land cover classification using Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs). The classification task aims to predict the category of land based on satellite or aerial images

This project aims to develop a Hybrid architecture of RNN and CNN to improve classification accuracy on an Image dataset. The Hybrid architecture had only 20% of trainable parameters compared to baseline CNN and rendered a classification accuracy of 86%.

Showcasing advanced deep learning projects utilizing CNNs and RNNs for tasks like image classification, customer churn prediction, and fake news detection. Demonstrates expertise in data preprocessing, model building, training, and evaluation to solve real-world problems.

A collection of Deep Learning projects and implementations using Python, TensorFlow, and Keras. Covers key concepts like Artificial Neural Networks (ANN), Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), and Transfer Learning. Includes image classification, NLP, and real-world AI applications with model training, evaluation.

It is well known that we can detect and classify diseases using the blood cells. Here we use different algorithms to process the data which the images of blood cells thereby we analyze it and generate the result(disease) accordingly by using classification techniques like RNN, CNN, SVM etc. So, by using ML techniques we can detect the disease by using neural networks with image classification. These results reveal that comparison between other Convolutional neural network our proposed neural network model is more efficient and accurate in classifying blood cell images. Based on the significance of blood cell classification in the diagnosis, researchers have proposed many algorithms to classify blood cells.

This experiment is the classification of human activities using a 2D pose time series dataset and an LSTM RNN. The idea is to prove the concept that using a series of 2D poses, rather than 3D poses or a raw 2D images, can produce an accurate estimation of the behaviour of a person or animal.

The world we live in today is filled with high-pressure jobs and education, and without the pressure, education, or employment, building a future is nearly impossible. Mental and emotional issues/disorders become pretty natural in such an environment. Some emotional/mental disorders cause problems in a person's work, and some are even found fueling the human being to excellency. The most uneasily found condition experienced since childhood or as an adult is attention deficit hyperactivity disorder (ADHD). This article deals with the use of Machine Learning (ML) techniques or algorithms to classify ADHD patients from non-ADHD patients. Various algorithms like Support Vector Machine, Decision Tree, Binary Classification can be used when the dataset is primarily of the text type; at the same time, the use of CNNs and RNNs can be made when the dataset is of image type for the classification. As for the Treatment part, various Cognitive Behavioral Therapies are provided online, and there are gaming apps that help train a patient. Concerning cognitive behavior, there are also Meditation and focus developing apps generated, which are studied and surveyed to suggest the best type for diagnosing ADHD.