AITemplate is a Python framework which renders neural network into high performance CUDA/HIP C++ code. Specialized for FP16 TensorCore (NVIDIA GPU) and MatrixCore (AMD GPU) inference.

Lightning fast C++/CUDA neural network framework

Several simple examples for popular neural network toolkits calling custom CUDA operators.

Komputation is a neural network framework for the Java Virtual Machine written in Kotlin and CUDA C.

Convolutional Neural Network with CUDA (MNIST 99.23%)

Convolutional neural networks C++ framework with CPU and GPU (CUDA) backends

Computer vision container that includes Jupyter notebooks with built-in code hinting, Anaconda, CUDA 11.8, TensorRT inference accelerator for Tensor cores, CuPy (GPU drop in replacement for Numpy), PyTorch, PyTorch geometric for Graph Neural Networks, TF2, Tensorboard, and OpenCV for accelerated workloads on NVIDIA Tensor cores and GPUs.

Spiking Neural Networks in C++ with strong GPU acceleration through CUDA

A GPU (CUDA) based Artificial Neural Network library

Simple neural network implementation using CUDA technology. It is an educational implementation.

A CUDA Extension of Neural Network Libraries

Neural network from scratch in CUDA/C++

autograd mir and CUDA library for dynamic neural networks in D.

Deep Learning framework in C++/CUDA that supports symbolic/automatic differentiation, dynamic computation graphs, tensor/matrix operations accelerated by GPU and implementations of various state-of-the-art graph neural networks and other Machine Learning models including Covariant Compositional Networks For Learning Graphs [Risi et al]

A GPGPU based Spiking Neural Network (SNN) designed to be as transparent and easy to modify as possible. Written in C++/CUDA.

High-performance CUDA kernels for equivariant graph neural networks (MACE, NequIP, Allegro). 10-20x faster than e3nn.

Image Classifier Going forward, AI algorithms will be incorporated into more and more everyday applications. For example, you might want to include an image classifier in a smartphone app. To do this, you'd use a deep learning model trained on hundreds of thousands of images as part of the overall application architecture. A large part of software development in the future will be using these types of models as common parts of applications. In this project, you'll train an image classifier to recognize different species of flowers. You can imagine using something like this in a phone app that tells you the name of the flower your camera is looking at. In practice, you'd train this classifier, then export it for use in your application. We'll be using this dataset of 102 flower categories. When you've completed this project, you'll have an application that can be trained on any set of labelled images. Here your network will be learning about flowers and end up as a command line application. But, what you do with your new skills depends on your imagination and effort in building a dataset. This is the final Project of the Udacity AI with Python Nanodegree Prerequisites The Code is written in Python 3.6.5 . If you don't have Python installed you can find it here. If you are using a lower version of Python you can upgrade using the pip package, ensuring you have the latest version of pip. To install pip run in the command Line python -m ensurepip -- default-pip to upgrade it python -m pip install -- upgrade pip setuptools wheel to upgrade Python pip install python -- upgrade Additional Packages that are required are: Numpy, Pandas, MatplotLib, Pytorch, PIL and json. You can donwload them using pip pip install numpy pandas matplotlib pil or conda conda install numpy pandas matplotlib pil In order to intall Pytorch head over to the Pytorch site select your specs and follow the instructions given. Viewing the Jyputer Notebook In order to better view and work on the jupyter Notebook I encourage you to use nbviewer . You can simply copy and paste the link to this website and you will be able to edit it without any problem. Alternatively you can clone the repository using git clone https://github.com/fotisk07/Image-Classifier/ then in the command Line type, after you have downloaded jupyter notebook type jupyter notebook locate the notebook and run it. Command Line Application Train a new network on a data set with train.py Basic Usage : python train.py data_directory Prints out current epoch, training loss, validation loss, and validation accuracy as the netowrk trains Options: Set direcotry to save checkpoints: python train.py data_dor --save_dir save_directory Choose arcitecture (alexnet, densenet121 or vgg16 available): pytnon train.py data_dir --arch "vgg16" Set hyperparameters: python train.py data_dir --learning_rate 0.001 --hidden_layer1 120 --epochs 20 Use GPU for training: python train.py data_dir --gpu gpu Predict flower name from an image with predict.py along with the probability of that name. That is you'll pass in a single image /path/to/image and return the flower name and class probability Basic usage: python predict.py /path/to/image checkpoint Options: Return top K most likely classes: python predict.py input checkpoint ---top_k 3 Use a mapping of categories to real names: python predict.py input checkpoint --category_names cat_To_name.json Use GPU for inference: python predict.py input checkpoint --gpu Json file In order for the network to print out the name of the flower a .json file is required. If you aren't familiar with json you can find information here. By using a .json file the data can be sorted into folders with numbers and those numbers will correspond to specific names specified in the .json file. Data and the json file The data used specifically for this assignemnt are a flower database are not provided in the repository as it's larger than what github allows. Nevertheless, feel free to create your own databases and train the model on them to use with your own projects. The structure of your data should be the following: The data need to comprised of 3 folders, test, train and validate. Generally the proportions should be 70% training 10% validate and 20% test. Inside the train, test and validate folders there should be folders bearing a specific number which corresponds to a specific category, clarified in the json file. For example if we have the image a.jpj and it is a rose it could be in a path like this /test/5/a.jpg and json file would be like this {...5:"rose",...}. Make sure to include a lot of photos of your catagories (more than 10) with different angles and different lighting conditions in order for the network to generalize better. GPU As the network makes use of a sophisticated deep convolutional neural network the training process is impossible to be done by a common laptop. In order to train your models to your local machine you have three options Cuda -- If you have an NVIDIA GPU then you can install CUDA from here. With Cuda you will be able to train your model however the process will still be time consuming Cloud Services -- There are many paid cloud services that let you train your models like AWS or Google Cloud Coogle Colab -- Google Colab gives you free access to a tesla K80 GPU for 12 hours at a time. Once 12 hours have ellapsed you can just reload and continue! The only limitation is that you have to upload the data to Google Drive and if the dataset is massive you may run out of space. However, once a model is trained then a normal CPU can be used for the predict.py file and you will have an answer within some seconds. Hyperparameters As you can see you have a wide selection of hyperparameters available and you can get even more by making small modifications to the code. Thus it may seem overly complicated to choose the right ones especially if the training needs at least 15 minutes to be completed. So here are some hints: By increasing the number of epochs the accuracy of the network on the training set gets better and better however be careful because if you pick a large number of epochs the network won't generalize well, that is to say it will have high accuracy on the training image and low accuracy on the test images. Eg: training for 12 epochs training accuracy: 85% Test accuracy: 82%. Training for 30 epochs training accuracy 95% test accuracy 50%. A big learning rate guarantees that the network will converge fast to a small error but it will constantly overshot A small learning rate guarantees that the network will reach greater accuracies but the learning process will take longer Densenet121 works best for images but the training process takes significantly longer than alexnet or vgg16 *My settings were lr=0.001, dropoup=0.5, epochs= 15 and my test accuracy was 86% with densenet121 as my feature extraction model. Pre-Trained Network The checkpoint.pth file contains the information of a network trained to recognise 102 different species of flowers. I has been trained with specific hyperparameters thus if you don't set them right the network will fail. In order to have a prediction for an image located in the path /path/to/image using my pretrained model you can simply type python predict.py /path/to/image checkpoint.pth Contributing Please read CONTRIBUTING.md for the process for submitting pull requests. Authors Shanmukha Mudigonda - Initial work Udacity - Final Project of the AI with Python Nanodegree

A Rust wrapper for Darknet, an open source neural network framework written in C and CUDA.

Deep neural network framework (C/C++/CUDA).

Machine Learning Tool with LSTM and MLP Neural Networks, with CUDA implementation.

Various version (CPU, CUDA_NAIVE, CUDA_TILED, GEMM) convolutional neural network implementations by Heechul Lim

autoTVM神经网络推理代码优化搜索演示，基于tvm编译开源模型centerface，并使用autoTVM搜索最优推理代码，　最终部署编译为c++代码，演示平台是cuda，可以是其他平台，例如树莓派，安卓手机，苹果手机．Thi is a demonstration of how to use autoTVM to search and optimize a neural network inference code. the main process of this program is , firstly use tvm to compile opensource model centerface , then use autotvm to auto search the best inference code for the compiled model

Example C++ CUDA implementation for training Neural Network on MNIST dataset

This is a c++ implementation of an LSTM Neural Network parallelized for a GPU using CUDA

Implement Neural Networks in Cuda from Scratch

Build CUDA Neural Network From Scratch

论文 Hinton等的论文 Matrix capsules with EM routing - Hinton, G. E., Sabour, S. and Frosst, N. (2018) Dynamic Routing Between Capsules - Sabour, S., Frosst, N. and Hinton, G.E. (2017) Transforming Auto-encoders - Hinton, G. E., Krizhevsky, A. and Wang, S. D. (2011) A parallel computation that assigns canonical object-based frames of reference. - Hinton, G.E. (1981) Shape representation in parallel systems - Hinton, G.E. (1981) Optimizing Neural Networks that Generate Images - Tijmen Tieleman’s disseration 其他论文 Capsule Network Performance on Complex Data - Xi, E., Bing, S. and Jin, Y. (2017) Accurate reconstruction of image stimuli from human fMRI based on the decoding model with capsule network architecture - Qiao, K., Zhang, C., Wang, L., Yan, B., Chen, J., Zeng, L. and Tong, L., (2018) An Optimization View on Dynamic Routing Between Capsules - Wang, D., Liu, E. (2018) CapsuleGAN: Generative Adversarial Capsule Network Ayush Jaiswal, Wael AbdAlmageed, Premkumar Natarajan. (2018) Spectral Capsule Networks - Bahadori, M. T. (2018) 博客 Max Pechyonkin的胶囊网络入门系列： 胶囊网络背后的直觉 胶囊如何工作 囊间动态路由算法 胶囊网络架构 基于TensorFlow实现胶囊网络 Debarko De的胶囊网络教程，包括注释详尽的胶囊网络实现代码 基于CUDA为胶囊网络实现TensorFlow定制操作 Jos van de Wolfshaar的文章，定制胶囊网络运算的CUDA支持 ISI新研究：胶囊生成对抗网络 用胶囊网络替换CNN作为GAN的判别网络，在MNIST数据集上取得了比卷积GAN更好的表现 Uncovering the Intuition behind Capsule Networks and Inverse Graphic Tanay Kothari的长篇教程 A Visual Representation of Capsule Connections in Dynamic Routing Between Capsules Mike Ross的胶囊网络示意图 Capsule Networks Are Shaking up AI — Here’s How to Use Them Nick Bourdakos的介绍 Capsule Networks Explained Kendrick Tan的解释 Understanding Dynamic Routing between Capsules (Capsule Networks) Jonathan Hui的教程，包括注释详尽的基于Keras的胶囊网络实现代码 Matrix capsules with EM routing Adrian Colyer关于EM路由的文章 Capsule Networks: A Glossary Sebastian Kwiatkowski的胶囊网络术语表 Overview of awesome articles 点评三篇胶囊网络教程 视频 Geoffrey Hinton’s talk: What is wrong with convolutional neural nets? - Geoffrey Hinton在MIT. Brain & Cognitive Sciences的演讲《卷积神经网络的问题在哪里？》 Capsule Networks (CapsNets) – Tutorial - “这视频棒极了。我本希望我能把胶囊解释得这么清楚。”Geoffrey Hinton Capsule networks: overview - 胶囊网络概览，包括向量和矩阵胶囊。 Overview of awesome videos 对以上3个视频的点评。 Capsule Networks: An Improvement to Convolutional Networks Siraj Raval介绍胶囊网络的视频 动态路由实现 官方实现 Sarasra/models 《Dynamic Routing Between Capsules》论文所用的代码 TensorFlow alisure-ml/CapsNet bourdakos1/capsule-networks etendue/CapsNet_TF InnerPeace-Wu/CapsNet-tensorflow jaesik817/adv_attack_capsnet jostosh/capsnet JunYeopLee/capsule-networks laodar/tf_CapsNet leoniloris/CapsNet naturomics/CapsNet-Tensorflow rrqq/CapsNet-tensorflow-jupyter thibo73800/capsnet-traffic-sign-classifier tjiang31/CapsNet winwinJJiang/capsNet-Tensorflow PyTorch acburigo/CapsNet adambielski/CapsNet-pytorch AlexHex7/CapsNet_pytorch aliasvishnu/Capsule-Networks-Notebook-MNIST andreaazzini/capsnet.pytorch cedrickchee/capsule-net-pytorch dragen1860/CapsNet-Pytorch gram-ai/capsule-networks higgsfield/Capsule-Network-Tutorial laubonghaudoi/CapsNet_guide_PyTorch leftthomas/CapsNet nishnik/CapsNet-PyTorch tonysy/CapsuleNet-PyTorch Ujjwal-9/CapsNet Keras fengwang/minimal-capsule gusgad/capsule-GAN mitiku1/Emopy-CapsNet ruslangrimov/capsnet-with-capsulewise-convolution streamride/CapsNet-keras-imdb sunxirui310/CapsNet-Keras theblackcat102/dynamic-routing-capsule-cifar XifengGuo/CapsNet-Keras XifengGuo/CapsNet-Fashion-MNIST Chainer soskek/dynamic_routing_between_capsules Torch mrkulk/Unsupervised-Capsule-Network MXNet AaronLeong/CapsNet_Mxnet GarrickLin/Capsnet.Gluon Soonhwan-Kwon/capsnet.mxnet CNTK Southworkscom/CapsNet-CNTK Lasagne DeniskaMazur/CapsNet-Lasagne Matlab yechengxi/LightCapsNet R dfalbel/capsnet JavaScript alseambusher/capsnet.js Vulcan moothyknight/CapsNet-for-Graphics-Rendering-Optimization EM路由实现 TensorFlow gyang274/capsulesEM www0wwwjs1/Matrix-Capsules-EM-Tensorflow PyTorch shzygmyx/Matrix-Capsules-pytorch 其他资源 Capsule Networks discussion Facebook讨论组 CapsNet-Tensorflow CapsNet-Tensorflow的gitter.im讨论组 Will capsule networks replace neural networks? Quora问答“胶囊网络会取代神经网络吗？” Could GANs work with Hinton’s capsule theory? Quora问答“GAN可以应用Hinton的胶囊理论吗？” Dynamic Routing Between Capsules Kyuhwan Jung对论文《Dynamic routing between Capsules》的评论（slideshare）

pyCUDA implementation of forward propagation for Convolutional Neural Networks

Reference CUDA implementation of training a small Bayesian neural network (BNN) using MCMC

Python/CUDA code to create a spiking neural network that can encode and traverse the Wordnet lexical database.