Jupyter Notebook is a pretty nifty tool that you can utilize in your day to day activities. To explain the benefits of Jupyter Notebook, we will share how we are using it to solve our regular puzzles at Elucidata.But before we deep dive into our specific usage, let’s get some context around Jupyter Notebooks.
Jupyter Notebook is an open-source web application that allows a user, scientific researcher, scholar or analyst to create and share the document called the Notebook, containing live codes, documentation, graphs, plots, and visualizations.Jupyter Notebook provides support for 40+ programming languages to the users including the most frequently used programming languages – Python, R, Julia to name a few. It allows the user to download the notebook in various file formats like PDF, HTML, Python, Markdown or an .ipynb file.
Due to Jupyter Notebook’s multi-programming support, huge feature availability and rapidly growing popularity among the community, it has become a standard for all sorts of analysis, visualizations, rapid prototyping, ML and various code practices.
Any person who is a data scientist, data engineer, data analyst, machine learning scientist, research scholar, scientific researchers or a general user who wants to do any sort of scientific computation, data processing or visualization related work can use the Jupyter Notebook.
Fig: [1] Image depicting the base components of the Jupyter NotebookHow a Jupyter Notebook works – a user interacting with the component of the Jupyter Notebook runs the code and stores the code output together with markdown notes in an editable doc called a notebook.When the user saves their notebook file, it is sent from the user browser to the notebook server. It is then saved on the disk as a JSON file with a .ipynb extension. The Jupyter Notebook server is responsible for saving, loading and editing the user notebooks if the kernel is still not present.
We, at Elucidata, are working on this project to develop new features and services on top of a traditional Jupyter Notebook, to facilitate our end users to have the best user experience.We have worked on creating a Jupyter Notebook with a brand new and elegant UI, and new custom functionalities. We are not leaving any stone unturned to make it the best notebook experience a user can have.We are dedicated to this project to make it, what a data scientist, data engineer or data scholar would want on our platform.
We introduced the Jupyter Notebooks in the eco-system of our platform – Polly™, to support the manipulation, visualization and the programming of the end result of the built-in workflows. Later, we leveraged the functionality of the Jupyter Notebook and combined it with the JupyterHub architecture to extend the functionalities for the following use cases:
Supporting such use cases, require a quite scalable and supporting infrastructure. Let’s walk through some of the components of our Jupyter Notebook System.
Why we chose JupyterHub + Docker?We didn’t want our users to fight over the correct package version installations and their dependency management for work. We wanted every user i.e. data scientists, data engineers, or data analysts, to have an identical reproducible environment with the same library and same datasets. In fact, the same version of everything.If we allow them to install on their own pods, it would lead to different environment versions depending on what workflow they are using for the package installer.A fully hosted environment makes sure that everybody has the same seamless starting point.
Get the latest insights on Biomolecular data and ML