PyRun Documentation

Appearance

FAQ: Frequently Asked Questions 🤔 ​

Have questions about PyRun? Find answers to common queries here. If you don't see your question listed, please reach out!

General / About PyRun ​

Q: What is PyRun? A: PyRun is a cloud platform designed to simplify running Python code, especially for data processing, AI/ML, and distributed computing tasks. It automates infrastructure management, runtime configuration, and provides an integrated environment, allowing you to focus on your code, not cloud complexities. Learn more here: What is PyRun?

Q: Who is PyRun for? A: PyRun is ideal for data scientists, researchers, machine learning engineers, developers, and businesses who need to run computationally intensive Python tasks in the cloud without becoming cloud infrastructure experts.

Q: How is PyRun different from Coiled? A: While both platforms help scale Python, PyRun offers a more integrated, VS Code-like web interface, native support for both Dask and Lithops (FaaS), automated runtime management, and a broader focus on various workflows including AI/ML. Benchmark data also shows significant cost and speed advantages in specific FaaS and Dask scenarios. Coiled focuses primarily on managed Dask cluster deployment. See a detailed comparison: PyRun vs. Coiled.

Q: How is PyRun different from just using Lithops or Dask directly? A: PyRun provides a managed layer on top of these powerful tools. It handles the complex setup, configuration (like Lithops backends for AWS and IBM Cloud), automatic runtime building via Runtime Management, provides an integrated development environment, real-time monitoring, and simplifies the overall user experience. You get the power of the frameworks without the setup overhead.

Q: Is PyRun free? How does pricing work? A: PyRun itself is currently offered free of charge during its initial phase (paid plans are anticipated in the future). However, the actual cloud computing resources (like AWS EC2 instances, Lambda functions, S3 storage, or IBM Code Engine resources) used by your PyRun jobs are billed directly to your connected cloud provider account. PyRun helps you use those resources efficiently but doesn't replace the cloud provider's billing.

Getting Started / Setup ​

Q: What do I need to get started with PyRun? A: You'll need an email address to sign up for PyRun and an AWS account to connect for running your jobs. Follow our Account Setup guide.

Q: Why do I need to connect my AWS account? A: PyRun doesn't run code on its own servers. It orchestrates the execution within your secure cloud environment (AWS). Connecting your account allows PyRun to create and manage the necessary resources (like compute instances, serverless functions, storage) on your behalf, ensuring you maintain control and transparency over costs.

Q: Can multiple PyRun users share a single AWS account? A: Currently, due to the way CloudFormation stack names work in AWS during the Account Setup, each linked AWS account can generally only be associated with a single PyRun user. We are aware of this limitation and are exploring solutions for multi-user support within a single AWS account for future releases.

Technical / Usage ​

Q: What cloud providers does PyRun support? A: Currently, PyRun integrates with Amazon Web Services (AWS) and has growing support for IBM Cloud (via the IBM Code Engine backend in Lithops/Dask). Support for Google Cloud Platform (GCP) and Microsoft Azure is planned on our Roadmap.

Q: What frameworks does PyRun support? A: PyRun offers first-class, integrated support for Lithops and Dask. It also supports Ray and Cubed through pre-built Pipelines.

Q: Does PyRun support AI and Machine Learning workloads? A: Absolutely! PyRun is an excellent platform for AI/ML. It provides the scalable compute and flexible environment management needed for the entire ML lifecycle. We offer pre-built AI Pipelines for use cases like audio recognition, image classification, distributed ML with Dask-ML, and LLM execution with Ollama. You can easily manage complex environments for libraries like TensorFlow, PyTorch, and scikit-learn using our Runtime Management. For a deeper dive, check out our AI Features page.

Q: How do I run my code on PyRun? A: After setting up your account and creating a project (Getting Started), you typically write or upload your Python code within your PyRun workspace and click the "Run" button. PyRun handles packaging, resource provisioning, execution, and monitoring. See the Basic Workflow.

Q: Where do my job outputs go? A: This depends on your code. Outputs might be saved as files within your PyRun workspace file browser, printed to the job logs (viewable in monitoring), or written to cloud storage (like AWS S3) if your code is configured to do so.

Runtime & Customization ​

Q: How do I install additional Python packages? A: You must define your dependencies within the .pyrun/environment.yml file (for Conda/pip packages) or by using a Dockerfile in the .pyrun/ directory. PyRun automatically rebuilds the runtime when these files change. Installing packages via the workspace terminal (pip install ...) will not make them available in the persistent runtime used for job execution. See Runtime Management for details.

Q: What's the difference between environment.yml and Dockerfile for runtime customization? A: environment.yml is simpler and recommended for managing just Python packages using Conda and pip. Dockerfile offers more control, allowing you to specify the base OS image, install system-level dependencies, and have fine-grained control over the entire environment setup. Both are covered in Runtime Management.

Q: Why do I need $LITHOPS in my environment.yml when using Lithops? A: PyRun provides specific optimizations and integrations for Lithops (like monitoring and backend configuration). Including $LITHOPS ensures you use the PyRun-compatible version, enabling these features. This is usually handled automatically when you create a Lithops workspace or template, but it's important to keep if manually editing. See the notes in Runtime Management.

Features ​

Q: What are Templates? A: Templates are pre-configured starting points for your workspaces, providing a basic environment and example code for specific tools like Lithops or Dask. They help you get started quickly after the initial setup. See Templates.

Q: What are Pipelines? A: Pipelines are more advanced, ready-to-run examples showcasing real-world use cases using PyRun with tools like Lithops, Dask, Ray, and Cubed. They are great for learning by example and seeing PyRun in action. See Pipelines.

Q: What is the Data Cockpit? A: The Data Cockpit is a feature (often used as a widget within a workspace) designed to help you manage, select, and partition data sources (like from S3) for easier integration with your compute jobs. See Data Cockpit.

Q: What are Core Concepts I should know? A: Understanding terms like Workspace, Runtime, Job, Task, Template, and Pipeline is helpful. Check out the Core Concepts page.

Support & Community ​

Q: Where can I report bugs or suggest features? A: We appreciate your feedback! Please report bugs or suggest features via our GitHub Issue Tracker.

Q: Where can I get help if I'm stuck? A: The best places to get help are:

  • GitHub Issues: For bug reports or specific technical issues, please use our GitHub Issue Tracker.
  • Our Community Discord Server: Join the conversation, ask questions, and interact with other users and the development team here: PyRun Discord Server

Can't find the answer you're looking for?