Deep Learning and its Tools

In past few years, deep artificial neural networks have won many contests in pattern recognition and machine learning. This is a result of the past research done and after collecting and summarising relevant work requirement in past decades. Deep Learners are distinguished by the depth of their credit assignment paths, which depends on the chains of possibly learning, links between actions and their effects. The present survey, however, will focus on the narrower, but now commercially important, a subfield of Deep Learning (DL) in Artificial Neural Networks (NNs). Talking about neural networks it consists of many connected sensors called neurons which produce a sequence of real-valued activations. Input neurons get activated through sensors perceiving the environment, other neurons get activated through weighted connections from previously active neurons. If we see the problem of how the neurons are connected, this may require long casual chains of computational stages. Basically, deep learning is all about how efficiently you assign credit across many such stages.
Deep Learning is emerging with a scope of great creativity in the latest techniques, for example:

  • Deep learning models are used for very difficult problems, like colourizing images and videos based on the context in the scene.
  • Deep learning models are used in new bold ways, such as cutting the head off a network trained on one problem and using it for a completely different problem and getting impressive results
  • Deep learning models’ combinations are used to both identify objects in photography and then generate textual descriptions of those objects

Many big tech companies are now researching to embed deep learning and launch tools in the market. The trendy tool in the market is “Python”. Keras is a minimalist Python library for deep learning that can run on top of TensorFlow. The guiding principles for this as per a Google Engineer, François Chollet are:

  • Modularity: A model can be understood as a sequence or a graph alone. All the concerns of a deep learning model are discrete components that can be combined in arbitrary ways.
  • Minimalism: The library provides just enough to achieve an outcome, no frills and maximising readability.
  • Extensibility: New components are intentionally easy to add and use within the framework, intended for researchers to trial and explore new ideas.
  • Python: No separate model files with custom file formats. Everything is native Python.

Following are some of the deep learning libraries which are open source using various different permissive licenses.

  • Theano: Theano is a Python framework developed by the LISA group (now MILA) run by Yoshua Bengio at the University of Montreal for research and development into state of the art deep learning algorithms.
  • Torch: Torch (called Torch7 using an odd version numbering) is a Lua deep learning framework developed by Ronan Collobert, Clement Farabet and Koray Kavukcuoglu for research and development into deep learning algorithms. It was used and promoted by the CILVR Lab at NYU (home to Yann LeCun).
  • Caffe: Caffe is a Python deep learning library developed by Yangqing Jia at the Berkeley Vision and Learning Center for supervised computer vision problems.
  • DeepLearning4J: DeepLearning4J (or DL4J for short) is a Deep Learning framework developed in Java (and JVM languages) by Adam Gibson for commercial deep learning projects.

There are many other excellent libraries and platforms. Some more honourable mentions include TensorFlow by Google, darch in R, Convnet.js in JavaScript (for learning), Mocha in Julia, CNTK by Microsoft and H2O Web API.

Ritesh Kanjee has over 7 years in Printed Circuit Board (PCB) design as well in image processing and embedded control. He completed his Masters Degree in Electronic engineering and published a paper for IEEE called Vision-based adaptive Cruise control using Pattern matching (on Google Scholar). His work was implemented in LabVIEW. He works as an Embedded Electronic Engineer in defence research. He has experience in FPGA design with programming in both VHDL and Verilog.