From de99650b486f5f4ecbc1b2d5f1b94861f43ee6d1 Mon Sep 17 00:00:00 2001 From: "promptless[bot]" <179508745+promptless[bot]@users.noreply.github.com> Date: Tue, 19 Aug 2025 21:01:22 +0000 Subject: [PATCH 1/4] Documentation updates from Promptless --- pods/templates/manage-templates.mdx | 134 ++++++++++++++++++++++++---- pods/templates/overview.mdx | 86 +++++++----------- pods/templates/secrets.mdx | 130 +++++++++++++++++++-------- 3 files changed, 238 insertions(+), 112 deletions(-) diff --git a/pods/templates/manage-templates.mdx b/pods/templates/manage-templates.mdx index 6e854333..09e413f0 100644 --- a/pods/templates/manage-templates.mdx +++ b/pods/templates/manage-templates.mdx @@ -1,28 +1,95 @@ --- -title: "Manage Pod Templates" +title: "Manage Pod templates" --- -## Explore Templates +## Explore templates -You can explore Templates managed by Runpod and Community Templates in the **[Explore](https://www.console.runpod.io/explore)** section of the Web interface. +You can discover and use existing templates through the Runpod console: -You can explore Templates managed by you or your team in the **[Templates](https://www.console.runpod.io/user/templates)** section of the Web interface. +**Browse all templates**: Visit the **[Explore](https://www.console.runpod.io/explore)** section to find official templates maintained by Runpod and community templates created by other users. -Learn to create your own Template in the following section. +**Manage your templates**: Access templates you've created or that are shared within your team in the **[My Templates](https://www.console.runpod.io/user/templates)** section. -## Creating a Template + +Runpod does not maintain community templates. If you encounter issues, contact the template creator directly or seek help on the [community Discord](https://discord.gg/runpod). + -Templates are used to launch images as a Pod: within a template, you define the required container disk size, volume, volume path, and ports needed. +## Creating a template -### Web interface +Creating a custom template allows you to package your specific configuration for reuse and sharing. Templates define all the necessary components to launch a Pod with your desired setup. - +### Template configuration options + +When creating a template, you'll configure several key components: + +#### Name +The display name for your template that will appear in the template browser. Choose a descriptive name that clearly indicates the template's purpose and contents. + +#### Container image +The Docker image that forms the foundation of your template. This can be: +- A public image from Docker Hub (e.g., `ubuntu:latest`, `pytorch/pytorch:latest`) +- A private image from your own registry (requires registry credentials) + +The container image is where the core functionality of your template resides, including all software packages, dependencies, and files needed for your workload. + +#### Template visibility +Choose whether your template is available to others: +- **Public template**: Available to all Runpod users in the community template section +- **Private template**: Only accessible to you or your team members + +#### Compute type +Templates are restricted to specific compute types and can only be used with matching hardware: +- **Nvidia GPU**: For GPU-accelerated workloads requiring CUDA support +- **AMD GPU**: For workloads optimized for AMD graphics processors +- **CPU**: For CPU-only workloads that don't require GPU acceleration + +#### Container Start Command +Customize the command that runs when your Pod starts. This overrides the default CMD instruction in your Docker container. You can specify: +- Simple bash commands: `bash -c 'mkdir /workspace && /start.sh'` +- JSON format with entrypoint and cmd: `{"cmd": ["python", "app.py"], "entrypoint": ["bash", "-c"]}` + +#### Registry credentials +If using a private container image, provide authentication credentials to access your private registry. This ensures Runpod can pull your image during Pod deployment. + +#### Storage configuration +Define the storage requirements for your template: + +**Container disk size**: The amount of storage allocated for the container's filesystem, including the operating system and installed packages. + +**Volume disk size**: Additional persistent storage that will be mounted to your Pod. This storage persists between Pod restarts and can be used for data, models, and other files you want to preserve. + +**Volume mount path**: The directory path where the persistent volume will be mounted inside the container (commonly `/workspace`). + +#### Network configuration +Configure network access for your template: + +**HTTP ports**: Ports that will be accessible via Runpod's HTTP proxy for web interfaces and APIs. These are automatically secured with HTTPS and accessible through Runpod's proxy URLs. + +**TCP ports**: Direct TCP port access for services that require raw TCP connections, such as SSH, databases, or custom protocols. + +#### Environment variables +Define key-value pairs that will be available as environment variables inside your Pod. These are useful for: +- Configuration settings specific to your application +- API keys and credentials (consider using [Secrets](/pods/templates/secrets) for sensitive data) +- Runtime parameters that customize your template's behavior + +### Creating templates via web interface + +The Runpod console provides an intuitive interface for template creation: + +1. Navigate to the **[Templates](https://www.console.runpod.io/user/templates)** section +2. Click **Create Template** +3. Fill in the configuration options described above +4. Test your template by deploying a Pod to ensure it works as expected +5. Save your template for future use + + -### cURL +### Creating templates via API -You can also create or modify a template using the Runpod API. +You can also create templates programmatically using the Runpod GraphQL API: ```sh curl --request POST \ @@ -31,16 +98,49 @@ curl --request POST \ --data '{"query": "mutation { saveTemplate(input: { containerDiskInGb: 5, dockerArgs: \"sleep infinity\", env: [ { key: \"key1\", value: \"value1\" }, { key: \"key2\", value: \"value2\" } ], imageName: \"ubuntu:latest\", name: \"Generated Template\", ports: \"8888/http,22/tcp\", readme: \"## Hello, World!\", volumeInGb: 15, volumeMountPath: \"/workspace\" }) { containerDiskInGb dockerArgs env { key value } id imageName name ports readme volumeInGb volumeMountPath } }"}' ``` -### Environment variables +This API call creates a template with: +- 5GB container disk +- Ubuntu latest image +- Custom environment variables +- HTTP port 8888 and TCP port 22 exposed +- 15GB persistent volume mounted at `/workspace` + +## Environment variables in templates -Environment variables in Runpod templates are key-value pairs that are accessible within your pod. Define a variable by setting a name with the `key` and then what it should contain with the `value`. +Environment variables provide a flexible way to configure your Pod's runtime behavior without modifying the container image. -Use environment variables to pass configuration settings and secrets to your container. For example, environment variables can store the path to a database or API keys used by your application. +### Defining environment variables - +Environment variables are key-value pairs that become available inside your Pod's container. When creating a template, you can define variables by specifying: +- **Key**: The environment variable name (e.g., `DATABASE_URL`, `API_KEY`) +- **Value**: The value assigned to that variable + + -Runpod also provides a set of predefined [environment variables](/pods/references/environment-variables) that provide information about the pod, such as the unique identifier for your pod (`RUNPOD_POD_ID`), the API key used to make Runpod API calls to the specific pod (`RUNPOD_API_KEY`), the name of the host server the pod is running on (`RUNPOD_POD_HOSTNAME`), and more. +### Use cases for environment variables + +Environment variables are particularly useful for: +- **Configuration settings**: Database connections, API endpoints, feature flags +- **Runtime parameters**: Model paths, batch sizes, processing options +- **Integration credentials**: API keys, authentication tokens (consider using [Secrets](/pods/templates/secrets) for sensitive data) +- **Application behavior**: Debug modes, logging levels, output formats + +### Runpod system environment variables + +Runpod automatically provides several [predefined environment variables](/pods/references/environment-variables) in every Pod: +- `RUNPOD_POD_ID`: Unique identifier for your Pod +- `RUNPOD_API_KEY`: API key for making Runpod API calls from within the Pod +- `RUNPOD_POD_HOSTNAME`: Hostname of the server running your Pod +- Additional system variables for GPU information, networking, and storage + +### Using Secrets in templates + +For sensitive information like passwords and API keys, use [Runpod Secrets](/pods/templates/secrets) instead of plain environment variables. Secrets are encrypted and can be referenced in your templates using the format: + +``` +{{ RUNPOD_SECRET_secret_name }} +``` -You can references [Secrets](/pods/templates/secrets) in your Pod templates. +This approach ensures sensitive data is properly protected while still being accessible to your Pod. diff --git a/pods/templates/overview.mdx b/pods/templates/overview.mdx index 228155f4..5a4271fb 100644 --- a/pods/templates/overview.mdx +++ b/pods/templates/overview.mdx @@ -2,86 +2,60 @@ title: "Overview" --- -Templates are Docker containers images paired with a configuration. +Pod templates are pre-configured Docker image setups that let you quickly spin up Pods without manual environment configuration. They're essentially deployment configurations that include specific models, frameworks, or workflows bundled together. -They are used to launch images as Pods, define the required container disk size, volume, volume paths, and ports needed. +Templates eliminate the need to manually set up environments, saving time and reducing configuration errors. For example, instead of installing PyTorch, configuring JupyterLab, and setting up all dependencies yourself, you can select a pre-configured template and have everything ready to go instantly. -You can also define environment variables within the Template. +## What Pod templates include -## Template types +Pod templates contain all the necessary components to launch a fully configured Pod: -There a few types of Templates: +**Container image**: The Docker image with all necessary software packages and dependencies. This is where the core functionality of the template is stored, i.e., the software package and any files associated with it. -* **Managed by Runpod**: Also known as official Templates; these templates are created and maintained by Runpod. +**Hardware specifications**: Container disk size, volume size, and mount paths that define the storage requirements for your Pod. -* **Custom Templates**: +**Network settings**: Exposed ports for services like web UIs or APIs. If the image has a server associated with it, you'll want to ensure that the HTTP and TCP ports are exposed as necessary. - * **Community Templates**: Custom Templates shared by the community. - * **Private Templates**: Custom Templates created by you or if using a team account, shared inside your team. +**Environment variables**: Pre-configured settings specific to the template that customize the behavior of the containerized application. -### Custom Templates +**Startup commands**: Instructions that run when the Pod launches, allowing you to customize the initialization process. -### Customizing Container Start Command +## Types of templates -You can customize the Docker command to run additional commands or modify the default behavior. +Runpod offers three types of templates to meet different needs: -The Docker command is specified in the **Container Start Command** field. +### Official templates -**Default Docker Command** +Official templates are curated by Runpod with proven demand and maintained quality. These templates undergo rigorous testing and are regularly updated to ensure compatibility and performance. Runpod provides full support for official templates. -The default Docker command is: +### Community templates -```sh -bash -c '/start.sh' -``` +Community templates are created by users and promoted based on community usage. These templates offer a wide variety of specialized configurations and cutting-edge tools contributed by the Runpod community. -This command runs the `/start.sh` script at the end of the container startup process. + +Runpod does not maintain community templates. If you encounter issues with a community template, please contact the template creator directly or ask for help on the [community Discord](https://discord.gg/runpod). + -You can customize the Docker command to run additional commands or modify the default behavior. +### Custom templates -For example, you can add a command to run before `/start.sh`: +You can create custom templates for your own specialized workloads. These can be private (visible only to you or your team) or made public for the community to use. -```sh -bash -c 'mkdir /testdir1 && /start.sh' -``` +## Why use Pod templates -This command creates a directory `/testdir1` before running `/start.sh`. +Templates provide significant advantages over manual Pod configuration: -**Using the `entrypoint` Field** +- **Time savings**: Popular templates include options for machine learning frameworks like PyTorch, image generation tools like Stable Diffusion, and development environments with Jupyter notebooks pre-installed. This eliminates hours of manual setup and dependency management. -You can also specify a JSON string with `cmd` and `entrypoint` as the keys. +- **Consistency**: Templates ensure that your development and production environments are identical, reducing "it works on my machine" issues. -The `entrypoint` field allows you to specify a command to run at the beginning of the container startup process. For example: +- **Best practices**: Official and popular community templates incorporate industry best practices for security, performance, and configuration. -```json -{ "cmd": ["echo foo && /start.sh"], "entrypoint": ["bash", "-c"] } -``` +- **Reduced errors**: Pre-configured templates minimize the risk of configuration mistakes that can lead to Pod startup failures or performance issues. -This command runs the `echo` command and then runs `/start.sh`. +## Container image considerations -**Important Considerations** +The container image is the foundation of any template and typically the most important component. Most users find it easiest to push their Docker images to Docker Hub for public access, though private registries are also supported. -When using the `entrypoint` field, be aware that the command will run twice: once as the entrypoint and again as part of the `cmd` field. +The image contains the core software package and any associated files needed for your workload. When creating or selecting a template, ensure that the image includes all necessary dependencies and is compatible with your intended use case. -This can cause issues if the command errors when run a second time. For example: - -```json -{ "cmd": ["mkdir /testdir11 && /start.sh"], "entrypoint": ["bash", "-c"] } -``` - -This command will run `mkdir` twice, which can cause errors if the directory already exists. - -**Tips and Examples** - -Here are some working examples to try in dev: - -* Command only: `bash -c 'mkdir /testdir1 && /start.sh'` -* Command only: `{"cmd": ["bash", "-c", "mkdir /testdir8 && /start.sh"]}` -* Command and Entrypoint: `{"cmd": ["test-echo-test-echo"], "entrypoint": ["echo"]}` -* Command and Entrypoint: `{"cmd": ["mkdir -p /testdir12 && /start.sh"], "entrypoint": ["bash", "-c"]}` - - - -Remember to use `mkdir -p` to avoid errors when creating directories. - - +If your template includes a server component (such as a web interface or API), make sure the appropriate HTTP and TCP ports are properly exposed in the template configuration to allow external access. diff --git a/pods/templates/secrets.mdx b/pods/templates/secrets.mdx index d651a93d..b0c64a96 100644 --- a/pods/templates/secrets.mdx +++ b/pods/templates/secrets.mdx @@ -2,75 +2,127 @@ title: "Secrets" --- -You can add Secrets to your Pods and templates. Secrets are encrypted strings of text that are used to store sensitive information, such as passwords, API keys, and other sensitive data. +Runpod Secrets provide a secure way to store and manage sensitive information such as API keys, passwords, and authentication tokens in your Pod templates. Secrets are encrypted at rest and can be safely referenced in your templates without exposing sensitive data in plain text. -## Create a Secret +## What are Runpod Secrets -You can create a Secret using the Runpod Web interface or the Runpod API. +Secrets are encrypted strings that store sensitive information separately from your template configuration. This approach offers several advantages: -1. Login into the Runpod Web interface and select [Secrets](https://www.console.runpod.io/user/secrets). +**Security**: Sensitive data is encrypted and never displayed in plain text once created, protecting against accidental exposure. -2. Choose **Create Secret** and provide the following: +**Reusability**: The same secret can be referenced across multiple templates and Pods without duplication. - i. **Secret Name**: The name of the Secret. +**Access control**: Secrets are tied to your account or team, ensuring only authorized users can access them. - ii. **Secret Value**: The value of the Secret. +**Audit trail**: Changes to secrets are tracked, providing visibility into when sensitive data is modified. - iii. **Description**: (optional) A description of the Secret. +## Creating a secret -3. Select **Create Secret**. +You can create secrets through the Runpod web interface to securely store sensitive information. - +### Using the web interface -Once a Secret is created, its value cannot be viewed. If you need to change the Secret, you must create a new one or [modify the Secret Value](#modify-a-secret). +1. Navigate to the [Secrets](https://www.console.runpod.io/user/secrets) section in the Runpod console +2. Click **Create Secret** to open the creation form +3. Provide the required information: + - **Secret Name**: A unique identifier for your secret (e.g., `huggingface_token`, `database_password`) + - **Secret Value**: The actual sensitive data you want to store + - **Description** (optional): A helpful description of what this secret contains or how it's used +4. Click **Create Secret** to save your encrypted secret + +Once a secret is created, its value cannot be viewed through the interface. This is a security feature that prevents accidental exposure of sensitive data. If you need to verify or change the value, you must modify the secret or create a new one. -## Modify a Secret +## Managing existing secrets + +### Modifying a secret value + +To update the value of an existing secret: + +1. Go to the [Secrets](https://www.console.runpod.io/user/secrets) section +2. Click on the name of the secret you want to modify +3. Click the configuration icon and select **Edit Secret Value** +4. Enter the new secret value +5. Click **Save Changes** to update the encrypted value + +### Viewing secret details -You can modify an existing Secret using the Runpod Web interface. +You can view metadata about your secrets without exposing the sensitive values: -1. Login into the Runpod Web interface and select [Secrets](https://www.console.runpod.io/user/secrets). -2. Select the name of the Secret you want to modify. -3. Select the configuration icon and choose **Edit Secret Value**. +1. Navigate to the [Secrets](https://www.console.runpod.io/user/secrets) section +2. Click on the secret name you want to inspect +3. Click the configuration icon and select **View Secret** - i. Enter your new Secret Value. -4. Select **Save Changes**. +This shows you the secret name, description, and creation date, but never the actual secret value. -## View Secret details +### Deleting a secret -You can view the details of an existing Secret using the Runpod Web interface. You can't view the Secret Value. +To permanently remove a secret: -1. Login into the Runpod Web interface and select [Secrets](https://www.console.runpod.io/user/secrets). -2. Select the name of the Secret you want to view. -3. Select the configuration icon and choose **View Secret**. +1. Go to the [Secrets](https://www.console.runpod.io/user/secrets) section +2. Click on the secret you want to delete +3. Click the configuration icon and select **Delete Secret** +4. Type the secret name to confirm deletion +5. Click **Confirm Delete** to permanently remove the secret -## Use a Secret in a Pod + +Deleting a secret is permanent and cannot be undone. Make sure no active templates or Pods are using the secret before deletion, as this will cause those deployments to fail. + -With your Secrets setup, you can now reference them in your Pods. +## Using secrets in Pod templates -You can reference your Secret directly or select it from the Web interface when creating or modifying a Pod template. +Once you've created secrets, you can reference them in your Pod templates to provide secure access to sensitive data. -**Reference your Secret directly** +### Direct reference method -You can reference your Secret directly in the [Environment Variables](/pods/references/environment-variables) section of your Pod template. To reference your Secret, reference it's key appended to the `RUNPOD_SECRET_` prefix. For example: +Reference your secrets directly in the environment variables section of your Pod template using the `RUNPOD_SECRET_` prefix followed by your secret name: ``` -{{ RUNPOD_SECRET_hello_world }} +{{ RUNPOD_SECRET_secret_name }} ``` -Where `hello_world` is the value of your Secret Name. +For example, if you created a secret named `huggingface_token`, you would reference it as: + + +{{ RUNPOD_SECRET_huggingface_token }} + + +This syntax tells Runpod to substitute the encrypted secret value when the Pod starts, making it available as an environment variable inside your container. + +### Web interface selection + +When creating or editing a Pod template through the web interface, you can also: + +1. Navigate to the environment variables section of your template +2. Use the secret selector to choose from your available secrets +3. The interface will automatically format the reference syntax for you + +### Best practices for using secrets + +**Naming conventions**: Use descriptive names that clearly indicate the secret's purpose (e.g., `openai_api_key`, `database_password`, `github_token`). + +**Environment variable mapping**: Map secrets to appropriately named environment variables in your templates: + +``` +API_KEY={{ RUNPOD_SECRET_openai_key }} +DATABASE_URL={{ RUNPOD_SECRET_db_connection }} +``` + +**Minimal exposure**: Only include secrets in templates that actually need them to reduce the attack surface. + +**Regular rotation**: Periodically update secret values, especially for long-lived credentials like API keys. + +## Common use cases + +Secrets are particularly valuable for: -**Select your Secret from the Web interface** +**API authentication**: Store API keys for services like OpenAI, Hugging Face, or cloud providers without hardcoding them in your templates. -Alternatively, you can select your Secret from the Web interface when creating or modifying a Pod template. +**Database credentials**: Securely provide database connection strings and passwords to your applications. -## Delete a Secret +**Model access tokens**: Store authentication tokens required to download gated models or datasets. -You can delete an existing Secret using the Runpod Web interface. +**Service integration**: Keep webhook URLs, service account keys, and other integration credentials secure. -1. Login into the Runpod Web interface and select [Secrets](https://www.console.runpod.io/user/secrets). -2. Select the name of the Secret you want to delete. -3. Select the configuration icon and choose **Delete Secret**. -4. Enter the name of the Secret to confirm deletion. -5. Select **Confirm Delete**. +**Development vs production**: Use different secrets for different environments while keeping the same template structure. From 1adeed169ca6ae467d10c492ac04f0a327efc71f Mon Sep 17 00:00:00 2001 From: Mo King Date: Wed, 20 Aug 2025 11:22:40 -0400 Subject: [PATCH 2/4] Refactor manage templates page --- docs.json | 1 - pods/templates/manage-templates.mdx | 133 ++++++++++++++-------------- pods/templates/overview.mdx | 18 ++-- pods/templates/secrets.mdx | 21 ++--- 4 files changed, 82 insertions(+), 91 deletions(-) diff --git a/docs.json b/docs.json index 145d433c..f7a3d03f 100644 --- a/docs.json +++ b/docs.json @@ -206,7 +206,6 @@ { "group": "Troubleshooting", "pages": [ - "references/troubleshooting/cuda-version-issue", "references/troubleshooting/leaked-api-keys", "references/troubleshooting/storage-full", "references/troubleshooting/troubleshooting-502-errors", diff --git a/pods/templates/manage-templates.mdx b/pods/templates/manage-templates.mdx index 09e413f0..e892d007 100644 --- a/pods/templates/manage-templates.mdx +++ b/pods/templates/manage-templates.mdx @@ -14,96 +14,98 @@ You can discover and use existing templates through the Runpod console: Runpod does not maintain community templates. If you encounter issues, contact the template creator directly or seek help on the [community Discord](https://discord.gg/runpod). -## Creating a template - Creating a custom template allows you to package your specific configuration for reuse and sharing. Templates define all the necessary components to launch a Pod with your desired setup. -### Template configuration options +## Template configuration options When creating a template, you'll configure several key components: -#### Name -The display name for your template that will appear in the template browser. Choose a descriptive name that clearly indicates the template's purpose and contents. - -#### Container image -The Docker image that forms the foundation of your template. This can be: -- A public image from Docker Hub (e.g., `ubuntu:latest`, `pytorch/pytorch:latest`) -- A private image from your own registry (requires registry credentials) +**Name:** The display name for your template that will appear in the template browser. Choose a descriptive name that clearly indicates the template's purpose and contents. -The container image is where the core functionality of your template resides, including all software packages, dependencies, and files needed for your workload. +**Container image:** The path to the Docker image that forms the foundation of your template. This is where the core functionality of your template resides, including all software packages, dependencies, and files needed for your workload. You can import the image from: +- A public registry like Docker Hub (e.g., `ubuntu:latest`, `pytorch/pytorch:latest`). +- Your own private registry (requires registry credentials). -#### Template visibility -Choose whether your template is available to others: -- **Public template**: Available to all Runpod users in the community template section -- **Private template**: Only accessible to you or your team members +**Template visibility:** Choose whether your template is available to others. Public templates are available to all Runpod users in the Explore section of the console, while private template are only accessible to you or your team members. -#### Compute type -Templates are restricted to specific compute types and can only be used with matching hardware: -- **Nvidia GPU**: For GPU-accelerated workloads requiring CUDA support +**Compute type:** Templates are restricted to specific compute types and can only be used with matching hardware: +- **NVIDIA GPU**: For GPU-accelerated workloads requiring CUDA support - **AMD GPU**: For workloads optimized for AMD graphics processors - **CPU**: For CPU-only workloads that don't require GPU acceleration -#### Container Start Command +**Container start command:** Customize the command that runs when your Pod starts. This overrides the default CMD instruction in your Docker container. You can specify: - Simple bash commands: `bash -c 'mkdir /workspace && /start.sh'` - JSON format with entrypoint and cmd: `{"cmd": ["python", "app.py"], "entrypoint": ["bash", "-c"]}` -#### Registry credentials -If using a private container image, provide authentication credentials to access your private registry. This ensures Runpod can pull your image during Pod deployment. - -#### Storage configuration -Define the storage requirements for your template: - -**Container disk size**: The amount of storage allocated for the container's filesystem, including the operating system and installed packages. - -**Volume disk size**: Additional persistent storage that will be mounted to your Pod. This storage persists between Pod restarts and can be used for data, models, and other files you want to preserve. +**Registry credentials:** If using a private container image, provide authentication credentials to access your private registry. This ensures Runpod can pull your image during Pod deployment. -**Volume mount path**: The directory path where the persistent volume will be mounted inside the container (commonly `/workspace`). +**Storage configuration: **Define the storage requirements for your template, including: +- **Container disk size**: The amount of storage allocated for the container's filesystem, including the operating system and installed packages. +- **Volume disk size**: Additional persistent storage that will be mounted to your Pod. This storage persists between Pod restarts and can be used for data, models, and other files you want to preserve. +- **Volume mount path**: The directory path where the persistent volume will be mounted inside the container (commonly `/workspace`). -#### Network configuration -Configure network access for your template: +**Network configuration:** Configure network access for your template: +- **HTTP ports**: Ports that will be accessible via Runpod's HTTP proxy for web interfaces and APIs. These are automatically secured with HTTPS and accessible through Runpod's proxy URLs. +- **TCP ports**: Direct TCP port access for services that require raw TCP connections, such as SSH, databases, or custom protocols. -**HTTP ports**: Ports that will be accessible via Runpod's HTTP proxy for web interfaces and APIs. These are automatically secured with HTTPS and accessible through Runpod's proxy URLs. +**Environment variables:** Define key-value pairs that will be available as environment variables inside your Pod. These are useful for: +- Configuration settings specific to your application. +- API keys and credentials (consider using [Secrets](/pods/templates/secrets) for sensitive data). +- Runtime parameters that customize your template's behavior. -**TCP ports**: Direct TCP port access for services that require raw TCP connections, such as SSH, databases, or custom protocols. +## Creating templates -#### Environment variables -Define key-value pairs that will be available as environment variables inside your Pod. These are useful for: -- Configuration settings specific to your application -- API keys and credentials (consider using [Secrets](/pods/templates/secrets) for sensitive data) -- Runtime parameters that customize your template's behavior + -### Creating templates via web interface + The Runpod console provides an intuitive interface for template creation: -1. Navigate to the **[Templates](https://www.console.runpod.io/user/templates)** section -2. Click **Create Template** -3. Fill in the configuration options described above -4. Test your template by deploying a Pod to ensure it works as expected -5. Save your template for future use +1. Navigate to the **[Templates](https://www.console.runpod.io/user/templates)** section. +2. Click **New Template**. +3. Fill in the configuration options described above. +4. Click **Save Template** to save it to your account. +5. Test your template by deploying a Pod to ensure it works as expected. - - - - -### Creating templates via API + -You can also create templates programmatically using the Runpod GraphQL API: + +You can also create templates programmatically using the Runpod REST API. For example: -```sh +```bash curl --request POST \ - --header 'content-type: application/json' \ - --url 'https://api.runpod.io/graphql?api_key=${YOUR_API_KEY}' \ - --data '{"query": "mutation { saveTemplate(input: { containerDiskInGb: 5, dockerArgs: \"sleep infinity\", env: [ { key: \"key1\", value: \"value1\" }, { key: \"key2\", value: \"value2\" } ], imageName: \"ubuntu:latest\", name: \"Generated Template\", ports: \"8888/http,22/tcp\", readme: \"## Hello, World!\", volumeInGb: 15, volumeMountPath: \"/workspace\" }) { containerDiskInGb dockerArgs env { key value } id imageName name ports readme volumeInGb volumeMountPath } }"}' + --url https://rest.runpod.io/v1/templates \ + --header 'Authorization: Bearer YOUR_API_KEY' \ + --header 'Content-Type: application/json' \ + --data '{ + "category": "NVIDIA", + "containerDiskInGb": 50, + "containerRegistryAuthId": "", + "dockerEntrypoint": [], + "dockerStartCmd": [], + "env": { + "ENV_VAR": "value" + }, + "imageName": "", + "isPublic": false, + "isServerless": false, + "name": "", + "ports": [ + "8888/http", + "22/tcp" + ], + "readme": "", + "volumeInGb": 20, + "volumeMountPath": "/workspace" +}' ``` -This API call creates a template with: -- 5GB container disk -- Ubuntu latest image -- Custom environment variables -- HTTP port 8888 and TCP port 22 exposed -- 15GB persistent volume mounted at `/workspace` +For more details, see the [API reference](/api-reference/templates/POST/templates). + + + + ## Environment variables in templates @@ -129,15 +131,14 @@ Environment variables are particularly useful for: ### Runpod system environment variables -Runpod automatically provides several [predefined environment variables](/pods/references/environment-variables) in every Pod: -- `RUNPOD_POD_ID`: Unique identifier for your Pod -- `RUNPOD_API_KEY`: API key for making Runpod API calls from within the Pod -- `RUNPOD_POD_HOSTNAME`: Hostname of the server running your Pod -- Additional system variables for GPU information, networking, and storage +Runpod automatically provides several [predefined environment variables](/pods/references/environment-variables) in every Pod, for example: +- `RUNPOD_POD_ID`: Unique identifier for your Pod. +- `RUNPOD_API_KEY`: API key for making Runpod API calls from within the Pod. +- `RUNPOD_POD_HOSTNAME`: Hostname of the server running your Pod. ### Using Secrets in templates -For sensitive information like passwords and API keys, use [Runpod Secrets](/pods/templates/secrets) instead of plain environment variables. Secrets are encrypted and can be referenced in your templates using the format: +For sensitive information like passwords and API keys, use [Runpod secrets](/pods/templates/secrets) instead of plain environment variables. Secrets are encrypted and can be referenced in your templates using the format: ``` {{ RUNPOD_SECRET_secret_name }} diff --git a/pods/templates/overview.mdx b/pods/templates/overview.mdx index 5a4271fb..d370aad5 100644 --- a/pods/templates/overview.mdx +++ b/pods/templates/overview.mdx @@ -1,5 +1,6 @@ --- title: "Overview" +description: "Streamline your deployments by creating Pod templates that bundle container images with hardware specs and network settings." --- Pod templates are pre-configured Docker image setups that let you quickly spin up Pods without manual environment configuration. They're essentially deployment configurations that include specific models, frameworks, or workflows bundled together. @@ -10,15 +11,11 @@ Templates eliminate the need to manually set up environments, saving time and re Pod templates contain all the necessary components to launch a fully configured Pod: -**Container image**: The Docker image with all necessary software packages and dependencies. This is where the core functionality of the template is stored, i.e., the software package and any files associated with it. - -**Hardware specifications**: Container disk size, volume size, and mount paths that define the storage requirements for your Pod. - -**Network settings**: Exposed ports for services like web UIs or APIs. If the image has a server associated with it, you'll want to ensure that the HTTP and TCP ports are exposed as necessary. - -**Environment variables**: Pre-configured settings specific to the template that customize the behavior of the containerized application. - -**Startup commands**: Instructions that run when the Pod launches, allowing you to customize the initialization process. +- **Container image**: The Docker image with all necessary software packages and dependencies. This is where the core functionality of the template is stored, i.e., the software package and any files associated with it. +- **Hardware specifications**: Container disk size, volume size, and mount paths that define the storage requirements for your Pod. +- **Network settings**: Exposed ports for services like web UIs or APIs. If the image has a server associated with it, you'll want to ensure that the HTTP and TCP ports are exposed as necessary. +- **Environment variables**: Pre-configured settings specific to the template that customize the behavior of the containerized application. +- **Startup commands**: Instructions that run when the Pod launches, allowing you to customize the initialization process. ## Types of templates @@ -45,11 +42,8 @@ You can create custom templates for your own specialized workloads. These can be Templates provide significant advantages over manual Pod configuration: - **Time savings**: Popular templates include options for machine learning frameworks like PyTorch, image generation tools like Stable Diffusion, and development environments with Jupyter notebooks pre-installed. This eliminates hours of manual setup and dependency management. - - **Consistency**: Templates ensure that your development and production environments are identical, reducing "it works on my machine" issues. - - **Best practices**: Official and popular community templates incorporate industry best practices for security, performance, and configuration. - - **Reduced errors**: Pre-configured templates minimize the risk of configuration mistakes that can lead to Pod startup failures or performance issues. ## Container image considerations diff --git a/pods/templates/secrets.mdx b/pods/templates/secrets.mdx index b0c64a96..120ff7dd 100644 --- a/pods/templates/secrets.mdx +++ b/pods/templates/secrets.mdx @@ -2,19 +2,16 @@ title: "Secrets" --- -Runpod Secrets provide a secure way to store and manage sensitive information such as API keys, passwords, and authentication tokens in your Pod templates. Secrets are encrypted at rest and can be safely referenced in your templates without exposing sensitive data in plain text. +Runpod secrets provide a secure way to store and manage sensitive information such as API keys, passwords, and authentication tokens in your Pod templates. Secrets are encrypted at rest and can be safely referenced in your templates without exposing sensitive data in plain text. ## What are Runpod Secrets Secrets are encrypted strings that store sensitive information separately from your template configuration. This approach offers several advantages: -**Security**: Sensitive data is encrypted and never displayed in plain text once created, protecting against accidental exposure. - -**Reusability**: The same secret can be referenced across multiple templates and Pods without duplication. - -**Access control**: Secrets are tied to your account or team, ensuring only authorized users can access them. - -**Audit trail**: Changes to secrets are tracked, providing visibility into when sensitive data is modified. +- **Security**: Sensitive data is encrypted and never displayed in plain text once created, protecting against accidental exposure. +- **Reusability**: The same secret can be referenced across multiple templates and Pods without duplication. +- **Access control**: Secrets are tied to your account or team, ensuring only authorized users can access them. +- **Audit trail**: Changes to secrets are tracked, providing visibility into when sensitive data is modified. ## Creating a secret @@ -30,9 +27,9 @@ You can create secrets through the Runpod web interface to securely store sensit - **Description** (optional): A helpful description of what this secret contains or how it's used 4. Click **Create Secret** to save your encrypted secret - + Once a secret is created, its value cannot be viewed through the interface. This is a security feature that prevents accidental exposure of sensitive data. If you need to verify or change the value, you must modify the secret or create a new one. - + ## Managing existing secrets @@ -84,9 +81,9 @@ Reference your secrets directly in the environment variables section of your Pod For example, if you created a secret named `huggingface_token`, you would reference it as: - +``` {{ RUNPOD_SECRET_huggingface_token }} - +``` This syntax tells Runpod to substitute the encrypted secret value when the Pod starts, making it available as an environment variable inside your container. From 29297d3f13d10a1f7caf091dceb720d43fa9ff56 Mon Sep 17 00:00:00 2001 From: Mo King Date: Thu, 21 Aug 2025 09:18:14 -0400 Subject: [PATCH 3/4] Add first draft of custom template tutorial --- docs.json | 1 + pods/templates/create-custom-template.mdx | 360 ++++++++++++++++++++++ pods/templates/manage-templates.mdx | 6 +- pods/templates/overview.mdx | 6 +- pods/templates/secrets.mdx | 8 +- references/faq.mdx | 12 + references/glossary.mdx | 7 +- 7 files changed, 387 insertions(+), 13 deletions(-) create mode 100644 pods/templates/create-custom-template.mdx diff --git a/docs.json b/docs.json index f7a3d03f..af77d4f7 100644 --- a/docs.json +++ b/docs.json @@ -152,6 +152,7 @@ "pages": [ "pods/templates/overview", "pods/templates/manage-templates", + "pods/templates/create-custom-template", "pods/templates/secrets" ] }, diff --git a/pods/templates/create-custom-template.mdx b/pods/templates/create-custom-template.mdx new file mode 100644 index 00000000..013b998f --- /dev/null +++ b/pods/templates/create-custom-template.mdx @@ -0,0 +1,360 @@ +--- +title: "Create a custom Pod template" +sidebarTitle: "Create a custom template" +description: "Learn how to extend official Runpod templates to create your own Pod templates." +tag: "NEW" +--- + +This tutorial shows you how to create custom Pod templates by extending official Runpod base images with additional Python dependencies and pre-baked ML models. You'll learn the complete workflow from Dockerfile creation to deployment and testing. + +Custom templates allow you to package your specific dependencies, models, and configurations into reusable Docker images that can be deployed as Pods. This approach saves time during Pod initialization and ensures consistent environments across deployments. + +## What you'll learn + +In this tutorial, you'll learn how to: + +- Create a Dockerfile that extends a Runpod base image. +- Add Python dependencies to an existing base image. +- Pre-bake ML models into your custom template. +- Build and push Docker images with the correct platform settings. +- Deploy and test your custom template as a Pod. + +## Requirements + +Before you begin, you'll need: + +- A [Runpod account](/get-started/manage-accounts). +- [Docker](https://www.docker.com/products/docker-desktop/) installed on your local machine. +- A [Docker Hub](https://hub.docker.com/) account for hosting your custom images. +- At least $5 in Runpod credits for testing. +- Basic familiarity with Docker and command-line operations. + +## Step 1: Create a custom Dockerfile + +First, you'll create a Dockerfile that extends a Runpod base image with additional dependencies: + +1. Create a new directory for your custom template: + +```bash +mkdir my-custom-template +cd my-custom-template +``` + +2. Create a new Dockerfile: + +```bash +touch Dockerfile +``` + +3. Open the Dockerfile in your preferred text editor and add the following content: + +```dockerfile +# Use the specified base image +FROM runpod/pytorch:2.8.0-py3.11-cuda12.8.1-cudnn-devel-ubuntu22.04 + +# Install additional Python dependencies +RUN pip install --no-cache-dir \ + transformers \ + accelerate + +# Don't specify a CMD here to use the base image's CMD +``` + +This extends the official Runpod PyTorch 2.8.0 base image, and installs two additional Python packages. This means that these packages will be automatically installed every time the Pod starts, so you won't need to run `pip install` again after Pod restarts. + + +When building custom templates, always start with a Runpod base image that matches your CUDA requirements. The base image includes essential components like the `/start.sh` script that handles Pod initialization. + + +To maintain access to packaged services (like JupyterLab and SSH over TCP), we avoid specifying a `CMD` or `ENTRYPOINT` in the Dockerfile. Runpod base images include a carefully configured startup script (`/start.sh`) that handles Pod initialization, SSH setup, and service startup. Overriding this can break Pod functionality. + +## Step 2: Add system dependencies + +If your application requires system-level packages, add them before the Python dependencies: + +```dockerfile +# Use the specified base image +FROM runpod/pytorch:2.8.0-py3.11-cuda12.8.1-cudnn-devel-ubuntu22.04 + +# Update package list and install system dependencies +RUN apt-get update && apt-get install -y \ + git \ + wget \ + curl \ + && rm -rf /var/lib/apt/lists/* + +# Install additional Python dependencies +RUN pip install --no-cache-dir \ + transformers \ + accelerate \ + datasets \ + torch-audio + +# Don't specify a CMD here to use the base image's CMD +``` + + +Always clean up package lists with `rm -rf /var/lib/apt/lists/*` after installing system packages to reduce image size. + + +## Step 3: Pre-bake ML models + +To reduce Pod setup overhead, you can pre-download models during the Pod initialization process. Here are two approaches: + +### Method 1: Simple model download script + +Create a Python script that downloads your model: + +1. Create a `download_model.py` file in your project directory: + +```python +from transformers import AutoTokenizer, AutoModelForCausalLM +import torch + +# Download and cache the model +model_name = "microsoft/DialoGPT-medium" +print(f"Downloading {model_name}...") + +tokenizer = AutoTokenizer.from_pretrained(model_name) +model = AutoModelForCausalLM.from_pretrained(model_name) + +print("Model downloaded and cached successfully!") +``` + +2. Update your Dockerfile to include and run this script: + +```dockerfile +# Use the specified base image +FROM runpod/pytorch:2.8.0-py3.11-cuda12.8.1-cudnn-devel-ubuntu22.04 + +# Install additional Python dependencies +RUN pip install --no-cache-dir \ + transformers \ + accelerate + +# Copy and run model download script +COPY download_model.py /tmp/download_model.py +RUN python /tmp/download_model.py && rm /tmp/download_model.py + +# Don't specify a CMD here to use the base image's CMD +``` + +### Method 2: Using the Hugging Face CLI + +For more control over model downloads, use the Hugging Face CLI: + +```dockerfile +# Use the specified base image +FROM runpod/pytorch:2.8.0-py3.11-cuda12.8.1-cudnn-devel-ubuntu22.04 + +# Install additional Python dependencies +RUN pip install --no-cache-dir \ + transformers \ + accelerate \ + huggingface_hub + +# Pre-download specific model files +RUN python -c "from huggingface_hub import snapshot_download; snapshot_download('microsoft/DialoGPT-medium', cache_dir='/root/.cache/huggingface')" + +# Don't specify a CMD here to use the base image's CMD +``` + + +Pre-baking large models will significantly increase your Docker image size and build time. Consider whether the faster Pod startup time justifies the larger image size for your use case. + + +## Step 4: Build and push your Docker image + +Now you're ready to build your custom image and push it to Docker Hub: + +1. Build your Docker image with the correct platform specification: + +```bash +docker build --platform=linux/amd64 -t my-custom-template:latest . +``` + + +The `--platform=linux/amd64` flag is crucial for Runpod compatibility. Runpod's infrastructure requires AMD64 architecture images. + + +2. Tag your image for Docker Hub (replace `YOUR_USERNAME` with your Docker Hub username): + +```bash +docker tag my-custom-template:latest YOUR_USERNAME/my-custom-template:latest +``` + +3. Push the image to Docker Hub: + +```bash +docker push YOUR_USERNAME/my-custom-template:latest +``` + + +If you haven't logged into Docker Hub from your command line, run `docker login` first and enter your Docker Hub credentials. + + +## Step 5: Create a Pod template in Runpod + +Next, create a Pod template using your custom Docker image: + +1. Navigate to the [Templates page](https://console.runpod.io/user/templates) in the Runpod console. +2. Click **New Template**. +3. Configure your template with these settings: + - **Name**: Give your template a descriptive name (e.g., "My Custom PyTorch Template"). + - **Container Image**: Enter your Docker Hub image name (e.g., `YOUR_USERNAME/my-custom-template:latest`). + - **Container Disk**: Set to at least 20 GB to accommodate your custom dependencies. + - **Volume Disk**: Set according to your storage needs (e.g., 20 GB). + - **Volume Mount Path**: Keep the default `/workspace`. + - **Expose HTTP Ports**: Add `8888` for JupyterLab access. + - **Expose TCP Ports**: Add `22` if you need SSH access. +4. Click **Save Template**. + +## Step 6: Deploy and test your custom template + +Now you're ready to deploy a Pod using your custom template to verify everything works correctly: + +1. Go to the [Pods page](https://console.runpod.io/pods) in the Runpod console. +2. Click **Deploy**. +3. Choose an appropriate GPU (make sure it meets the CUDA version requirements of your base image). +4. Click **Change Template** and select your custom template under **Your Pod Templates**. +5. Fill out the rest of the settings as desired, then click **Deploy On Demand**. +6. Wait for your Pod to initialize (this may take 5-10 minutes for the first deployment). + +## Step 7: Verify your custom template + +Once your Pod is running, verify that your customizations work correctly: + +1. Find your Pod on the [Pods page](https://console.runpod.io/pods) and click on it to open the connection menu. Click Jupyter Lab under HTTP Services to open JupyterLab. +2. Create a new Python notebook and test your pre-installed dependencies: + +```python +# Test that your custom packages are installed +import transformers +import accelerate +print(f"Transformers version: {transformers.__version__}") +print(f"Accelerate version: {accelerate.__version__}") + +# If you pre-baked a model, test loading it +from transformers import AutoTokenizer, AutoModelForCausalLM + +model_name = "microsoft/DialoGPT-medium" +tokenizer = AutoTokenizer.from_pretrained(model_name) +model = AutoModelForCausalLM.from_pretrained(model_name) + +print("Model loaded successfully!") +``` + +3. Run the cell to confirm everything is working as expected. + +## Advanced customization options + +### Setting environment variables + +You can set default environment variables in your template configuration: + +1. In the template creation form, scroll to **Environment Variables**. +2. Add key-value pairs for any environment variables your application needs: + - Key: `HUGGINGFACE_HUB_CACHE` + - Value: `/workspace/hf_cache` + +### Adding startup scripts + +To run custom initialization code when your Pod starts, create a startup script: + +1. Create a `startup.sh` file in your project directory: + +```bash +#!/bin/bash +echo "Running custom startup script..." +mkdir -p /workspace/models +echo "Custom startup complete!" +``` + +2. Add it to your Dockerfile: + +```dockerfile +# Copy startup script +COPY startup.sh /usr/local/bin/startup.sh +RUN chmod +x /usr/local/bin/startup.sh + +# Modify the start script to run our custom startup +RUN echo '/usr/local/bin/startup.sh' >> /start.sh +``` + +### Using multi-stage builds + +For complex applications, use multi-stage builds to reduce final image size: + +```dockerfile +# Build stage +FROM runpod/pytorch:2.8.0-py3.11-cuda12.8.1-cudnn-devel-ubuntu22.04 as builder + +# Install build dependencies +RUN apt-get update && apt-get install -y build-essential +RUN pip install --no-cache-dir some-package-that-needs-compilation + +# Final stage +FROM runpod/pytorch:2.8.0-py3.11-cuda12.8.1-cudnn-devel-ubuntu22.04 + +# Copy only the necessary files from builder +COPY --from=builder /usr/local/lib/python3.11/site-packages /usr/local/lib/python3.11/site-packages + +# Install runtime dependencies +RUN pip install --no-cache-dir transformers accelerate +``` + +## Troubleshooting + +Here are solutions to common issues when creating custom templates: + +### Build failures + +- **Platform mismatch**: Always use `--platform=linux/amd64` when building. +- **Base image not found**: Verify the base image tag exists on Docker Hub. +- **Package installation fails**: Check that package names are correct and available for the Python version in your base image. + +### Pod deployment issues + +- **Pod fails to start**: Check the Pod logs in the Runpod console for error messages. +- **Services not accessible**: Ensure you've exposed the correct ports in your template configuration. +- **CUDA version mismatch**: Make sure your base image CUDA version is compatible with your chosen GPU. + +### Performance issues + +- **Slow startup**: Consider pre-baking more dependencies or using a smaller base image. +- **Out of memory**: Increase container disk size or choose a GPU with more VRAM. +- **Model loading errors**: Verify that pre-baked models are in the expected cache directories. + +## Best practices + +Follow these best practices when creating custom templates: + +### Image optimization + +- Use `.dockerignore` to exclude unnecessary files from your build context. +- Combine RUN commands to reduce image layers. +- Clean up package caches and temporary files. +- Use specific version tags for dependencies to ensure reproducibility. + +### Security considerations + +- Don't include sensitive information like API keys in your Docker image. +- Use [Runpod Secrets](/pods/templates/secrets) for sensitive configuration. +- Regularly update base images to get security patches. + +### Version management + +- Tag your images with version numbers (e.g., `v1.0.0`) instead of just `latest`. +- Keep a changelog of what changes between versions. +- Test new versions thoroughly before updating production templates. + +## Next steps + +Now that you have a working custom template, consider these next steps: + +- **Automate builds**: Set up GitHub Actions or similar CI/CD to automatically build and push new versions of your template. +- **Share with team**: If you're using a team account, share your template with team members. +- **Create variations**: Build specialized versions of your template for different use cases (development vs. production). +- **Monitor usage**: Track how your custom templates perform in production and optimize accordingly. + +For more advanced template management, see the [Template Management API documentation](/api-reference/templates/POST/templates) to programmatically create and update templates. \ No newline at end of file diff --git a/pods/templates/manage-templates.mdx b/pods/templates/manage-templates.mdx index e892d007..5aa48b2f 100644 --- a/pods/templates/manage-templates.mdx +++ b/pods/templates/manage-templates.mdx @@ -11,7 +11,7 @@ You can discover and use existing templates through the Runpod console: **Manage your templates**: Access templates you've created or that are shared within your team in the **[My Templates](https://www.console.runpod.io/user/templates)** section. -Runpod does not maintain community templates. If you encounter issues, contact the template creator directly or seek help on the [community Discord](https://discord.gg/runpod). +Runpod does not maintain or provide customer support for community templates. If you encounter issues, contact the template creator directly or seek help on the [community Discord](https://discord.gg/runpod). Creating a custom template allows you to package your specific configuration for reuse and sharing. Templates define all the necessary components to launch a Pod with your desired setup. @@ -117,10 +117,6 @@ Environment variables are key-value pairs that become available inside your Pod' - **Key**: The environment variable name (e.g., `DATABASE_URL`, `API_KEY`) - **Value**: The value assigned to that variable - - - - ### Use cases for environment variables Environment variables are particularly useful for: diff --git a/pods/templates/overview.mdx b/pods/templates/overview.mdx index d370aad5..c6346371 100644 --- a/pods/templates/overview.mdx +++ b/pods/templates/overview.mdx @@ -1,6 +1,6 @@ --- title: "Overview" -description: "Streamline your deployments by creating Pod templates that bundle container images with hardware specs and network settings." +description: "Streamline your Pod deployments with templates, bundling prebuilt container images with hardware specs and network settings." --- Pod templates are pre-configured Docker image setups that let you quickly spin up Pods without manual environment configuration. They're essentially deployment configurations that include specific models, frameworks, or workflows bundled together. @@ -30,13 +30,15 @@ Official templates are curated by Runpod with proven demand and maintained quali Community templates are created by users and promoted based on community usage. These templates offer a wide variety of specialized configurations and cutting-edge tools contributed by the Runpod community. -Runpod does not maintain community templates. If you encounter issues with a community template, please contact the template creator directly or ask for help on the [community Discord](https://discord.gg/runpod). +Runpod does not maintain or provide customer support for community templates. If you encounter issues, contact the template creator directly or seek help on the [community Discord](https://discord.gg/runpod). ### Custom templates You can create custom templates for your own specialized workloads. These can be private (visible only to you or your team) or made public for the community to use. +To learn how to create your a custom template by extending an official Runpod base image, see [Create a custom Pod template](/pods/templates/create-custom-template). + ## Why use Pod templates Templates provide significant advantages over manual Pod configuration: diff --git a/pods/templates/secrets.mdx b/pods/templates/secrets.mdx index 120ff7dd..3e956f1c 100644 --- a/pods/templates/secrets.mdx +++ b/pods/templates/secrets.mdx @@ -1,5 +1,5 @@ --- -title: "Secrets" +title: "Manage secrets" --- Runpod secrets provide a secure way to store and manage sensitive information such as API keys, passwords, and authentication tokens in your Pod templates. Secrets are encrypted at rest and can be safely referenced in your templates without exposing sensitive data in plain text. @@ -15,9 +15,7 @@ Secrets are encrypted strings that store sensitive information separately from y ## Creating a secret -You can create secrets through the Runpod web interface to securely store sensitive information. - -### Using the web interface +You can create secrets through the Runpod web interface to securely store sensitive information: 1. Navigate to the [Secrets](https://www.console.runpod.io/user/secrets) section in the Runpod console 2. Click **Create Secret** to open the creation form @@ -122,4 +120,4 @@ Secrets are particularly valuable for: **Service integration**: Keep webhook URLs, service account keys, and other integration credentials secure. -**Development vs production**: Use different secrets for different environments while keeping the same template structure. +**Development vs. production**: Use different secrets for different environments while keeping the same template structure. diff --git a/references/faq.mdx b/references/faq.mdx index 57650632..b0419282 100644 --- a/references/faq.mdx +++ b/references/faq.mdx @@ -107,6 +107,18 @@ Data privacy is important to us at Runpod. Our Terms of Service prohibit hosts f You can run any Docker container available on any publicly reachable container registry. If you are not well versed in containers, we recommend sticking with the default run templates like our Runpod PyTorch template. However, if you know what you are doing, you can do a lot more! +### Does RunPod support Community Templates? + +**No, Runpod does not provide support for community templates.** Community templates are Docker container images created and shared by individual users and organizations in the Runpod community. While you can use these templates, you are responsible for ensuring they work for your use case. + +If you encounter issues with a community template: +- Contact the template creator directly for support. +- Check the template's documentation or repository for troubleshooting guides. +- Ask for help in the [Runpod Discord community](https://discord.gg/runpod). +- Consider creating your own custom template based on your specific needs. + +Runpod support can only assist with platform-related issues (Pod deployment, billing, network connectivity, etc.) and does not extend to the contents or functionality of community templates. + ### Can I run my own Docker daemon on Runpod? You can't currently spin up your own instance of Docker, as we run Docker for you! Unfortunately, this means that you cannot currently build Docker containers on Runpod or use things like Docker Compose. Many use cases can be solved by creating a custom template with the Docker image that you want to run. diff --git a/references/glossary.mdx b/references/glossary.mdx index f23cacdb..580c0891 100644 --- a/references/glossary.mdx +++ b/references/glossary.mdx @@ -51,7 +51,12 @@ These instances spin up in seconds using both public and private repositories. T ## Template -A Runpod template is a Docker container image paired with a configuration. +A [Runpod template](/pods/templates/overview) is a Docker container image paired with a configuration. Templates define the required container disk size, volume, volume paths, ports, and environment variables needed to launch Pods. + +There are three types of templates: +- **Official templates**: Official templates created and maintained by Runpod with full support. +- **Community templates**: User-contributed templates shared by the community (Runpod does not maintain these, or provide customer support for them). +- **Private templates**: Custom templates created by you or your team. ## SDKs From ae4237a82a00935b0906965e758d4c6a5a408906 Mon Sep 17 00:00:00 2001 From: Mo King Date: Thu, 21 Aug 2025 10:10:51 -0400 Subject: [PATCH 4/4] Update tutorial --- pods/templates/create-custom-template.mdx | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/pods/templates/create-custom-template.mdx b/pods/templates/create-custom-template.mdx index 013b998f..e861e182 100644 --- a/pods/templates/create-custom-template.mdx +++ b/pods/templates/create-custom-template.mdx @@ -2,10 +2,9 @@ title: "Create a custom Pod template" sidebarTitle: "Create a custom template" description: "Learn how to extend official Runpod templates to create your own Pod templates." -tag: "NEW" --- -This tutorial shows you how to create custom Pod templates by extending official Runpod base images with additional Python dependencies and pre-baked ML models. You'll learn the complete workflow from Dockerfile creation to deployment and testing. +This tutorial shows how to create custom Pod templates by extending official Runpod base images with additional Python dependencies and pre-baked ML models. You'll learn the complete workflow from Dockerfile creation to deployment and testing. Custom templates allow you to package your specific dependencies, models, and configurations into reusable Docker images that can be deployed as Pods. This approach saves time during Pod initialization and ensures consistent environments across deployments. @@ -15,7 +14,7 @@ In this tutorial, you'll learn how to: - Create a Dockerfile that extends a Runpod base image. - Add Python dependencies to an existing base image. -- Pre-bake ML models into your custom template. +- Pre-package ML models into your custom template. - Build and push Docker images with the correct platform settings. - Deploy and test your custom template as a Pod. @@ -105,7 +104,7 @@ To reduce Pod setup overhead, you can pre-download models during the Pod initial Create a Python script that downloads your model: -1. Create a `download_model.py` file in your project directory: +1. Create a file named `download_model.py` in the same directory as your Dockerfile: ```python from transformers import AutoTokenizer, AutoModelForCausalLM