We take a deep dive into the challenge of how serverless computing can be easily used for a broad range of scenarios, like high-performance computing, Monte Carlo simulations, Big Data pre-processing and molecular biology. We’ll focus on the user experience how to connect existing code and frameworks to serverless without the painful process of starting from scratch and or learning new skills. To achieve this, we present the open source Lithops (http://lithops.cloud) framework, that introduces serverless with minimal effort, and its new fusion with serverless computing brings automated scalability and the use of existing frameworks into the picture. Lithops, a novel toolkit that enables the transparent execution of unmodified, regular Python code against disaggregated cloud resources. Lithops supports hybrid execution environments of using Kubernetes, Apache OpenWhisk and any of the major serverless computing offerings in today’s market .
Lithops minimizes the learning curve for knowledgeable Python developers, keeps interfaces simple and consistent, and provides access transparency to disaggregated storage and memory resources in the cloud. Further, its multicloud-agnostic architecture ensures portability and overcomes vendor lock in. Altogether, this represents a significant step forward in the programmability of the cloud.
We present real use cases and examples with Lithops, including Monte Carlo simulations, data pre-processing and the use case of European Molecular Biology Laboratory (EMBL), and how Lithops framework allowed to process datasets which were previously out of reach, and without additional efforts for infrastructure maintenance, configuration and deployment.
Cloud and Big Data Expert