# Welcome to Denvr AI Cloud Docs! Denvr AI Services are built for AI workloads, offering training, inference, and deployment tools for AI professionals, including researchers, MLOps, and software and ML engineers.
AI clusters

Data center GPUs including NVIDIA H100, A100 & A40 with compute fabric up to 3.2Tbps of non-blocking InfiniBand.

Storage

High-performance direct attached and network filesystems to cache and persist data for your compute.

API reference

Automate provisioning and management of applications, virtual machines, and bare metal hosts.
Users can provision AI infrastructure in three primary ways: * [Applications](/docs/platform/applications) - managed Notebooks, IDEs, and private containers running on bare metal * [Virtual machines](/docs/overview/getting-started/launch-a-virtual-machine) - support for 1x, 2x, 4x, and 8x GPU configurations * [Bare metal ](/docs/platform/bare-metal)- direct access to host for peak performance # Getting started ### URLs Dashboard (UI): [https://console.cloud.denvrdata.com](https://console.cloud.denvrdata.com/) API: [https://api.cloud.denvrdata.com](https://api.cloud.denvrdata.com/) ### Denvr AI Cloud Demo and Test Drives Denvr can assist your onboarding decisions with guided demos by our Sales Engineers, as well as Test Drives. Test Drives typically give access to a reserved full-node but can be tailored to your requirements. [Please contact our team for details!](https://www.denvrdata.com/contact-our-team)
### New accounts setup To create a new Tenant account in Denvr AI Cloud, please register via the [Cloud Console](https://console.cloud.denvrdata.com/account/login). All new setup requests are reviewed manually to ensure we have capacity to fill your needs.
# Launch a virtual machine Explore the functionality of the Virtual Machines screen, which provides a comprehensive overview and control of your virtual machines, including creation, management, and monitoring features. ## Overview Navigate to the Virtual Machines overview screen by selecting the 'Virtual Machines' item directly from the left navigation menu. This view will display running and stopped instances.
## **Creating a new VM** To launch a new Virtual Machine instance, click the 'Create Virtual Machine' button:
Begin by entering a unique name for the instance and highlight the `on-demand` or `reserved` resource pool to use (if your tenant has reservations).
Select the operating system type and release version. Denvr Cloud can optionally install current stable releases of GPU drivers and required toolkits.
Finally, choose what types of network attached storage to use with your instance, and paste your SSH public key in the 'SSH Keys' box. Click 'Launch Instance' to begin provisioning.
You will be redirected back to the overview screen where the new VM appears. You will notice the VM status changing as it progresses from PENDING to ONLINE.
Once the Virtual Machine you created reaches ONLINE status, it is ready for use, and you can try pinging it throught the Public IP. ``` % ping 130.250.171.45 PING 130.250.171.45 (130.250.171.45): 56 data bytes 64 bytes from 130.250.171.45: icmp_seq=0 ttl=51 time=46.607 ms 64 bytes from 130.250.171.45: icmp_seq=1 ttl=51 time=43.375 ms 64 bytes from 130.250.171.45: icmp_seq=2 ttl=51 time=43.511 ms 64 bytes from 130.250.171.45: icmp_seq=3 ttl=51 time=44.528 ms 64 bytes from 130.250.171.45: icmp_seq=4 ttl=51 time=43.767 ms 64 bytes from 130.250.171.45: icmp_seq=5 ttl=51 time=44.102 ms 64 bytes from 130.250.171.45: icmp_seq=6 ttl=51 time=43.524 ms ^C --- 130.250.171.45 ping statistics --- 8 packets transmitted, 7 packets received, 12.5% packet loss round-trip min/avg/max/stddev = 43.375/44.202/46.607/1.050 ms ``` Connect to the instance using your SSH: ``` $ ssh -i ~/.ssh/id_rsa ubuntu@130.250.171.45 ``` {% hint style="info" %} You don't need to provide the -i parameter to SSH if your private key is the default key (id\_rsa) or your key is setup in `$HOME/.ssh/config` file. {% endhint %} You should see output similar to this: ``` kyle@kyles-macbook-pro ~ % ssh ubuntu@130.250.171.45 The authenticity of host '130.250.171.45 (130.250.171.45)' can't be established. ED25519 key fingerprint is SHA256:zDgLTz7gwIGPBf2uXg0YgVdLkEySO2g+fwLgS99nmdM. This key is not known by any other names Are you sure you want to continue connecting (yes/no/[fingerprint])? yes Warning: Permanently added '130.250.171.45' (ED25519) to the list of known hosts. Welcome to Ubuntu 22.04.4 LTS (GNU/Linux 5.15.0-101-generic x86_64) * Documentation: https://help.ubuntu.com * Management: https://landscape.canonical.com * Support: https://ubuntu.com/pro System information as of Wed Jul 3 14:39:36 UTC 2024 System load: 0.30029296875 Processes: 239 Usage of /: 0.3% of 457.75GB Users logged in: 0 Memory usage: 0% IPv4 address for enp2s0: 172.16.0.143 Swap usage: 0% Expanded Security Maintenance for Applications is not enabled. 0 updates can be applied immediately. Enable ESM Apps to receive additional future security updates. See https://ubuntu.com/esm or run: sudo pro status The list of available updates is more than a week old. To check for new updates run: sudo apt update The programs included with the Ubuntu system are free software; the exact distribution terms for each program are described in the individual files in /usr/share/doc/*/copyright. Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by applicable law. To run a command as administrator (user "root"), use "sudo ". See "man sudo_root" for details. ubuntu@vm-2024070310348308:~$ ``` ## **Managing VMs** The main screen for Virtual Machines serves as a control hub, allowing you to start, stop, delete, and view detailed information about each instance. Use the 3 vertical dots icon next to each VM for lifecycle actions: * Start - resume a stopped VM * Stop - stop a running VM. Resources are freed, while the root disk remains persisted * Delete - fully release a VM. This action can't be undone and removes all instance data
VM details are provided by selecting the instance Name:
## Usage metrics To track the real-time usage of your tenants' virtual machines, explore the 'GPU Utilization' screen. It offers a graphical and tabular representation of your Virtual Machines' usage, providing daily insights and performance data analysis for up to the last two months. Each utilization record begins at the moment a Virtual Machine is provisioned and continues until the Virtual Machine is stopped, ensuring accurate and continuous tracking of usage.
# Launch vLLM as a custom application This tutorial walks you through deploying language model inference endpoints on Denvr Dataworks using the Custom Application feature. We'll set up a [vLLM](https://docs.vllm.ai/) server hosting `mistralai/Mistral-7B-v0.1` with OpenAI-compatible APIs, leveraging Denvr Dataworks's GPU infrastructure and container orchestration capabilities. For detailed information about the Mistral-7B model, including its architecture, capabilities, and licensing terms, visit the [official model page on Hugging Face](https://huggingface.co/mistralai/Mistral-7B-v0.1). > **💡 Using Different Models**: To use a different model, replace `mistralai/Mistral-7B-v0.1` in the command override, select the appropriate hardware, accept any license agreements on the model's Hugging Face page, and update API examples accordingly. *** ### Understanding Custom Applications in Denvr Dataworks Denvr Dataworks's **Custom Application** feature is a powerful deployment method that gives you complete control over container configuration. Unlike other applications with predefined settings, custom applications allow you to: * **Bring Your Own Container**: Use any Docker image from public or private registries * **Reverse Proxy with API Authentication**: Configure an optional reverse proxy layer with API key authentication for secure access control * **Readiness Monitoring**: Configure a readiness port that gets pinged to verify your application is ready * **Command Override**: Replace the default container entry point with your own commands * **Environment Customization**: Set custom environment variables for your specific needs * **User Scripts**: Upload custom scripts that are mounted at `/etc/script/user-scripts` and executable from your command override > **ℹ️ Deployment Requirement**: Custom applications are currently only available to launch through reserved resource pools. On-demand resource pools are not yet supported for custom application deployments. ### Prerequisites For this tutorial you need: * A Denvr Dataworks account with **Custom Application** deployment capability (found through the **Application Catalog**) * A Hugging Face account with a User Access Token (you'll create this in the next section) * Command line access with `curl` for testing the deployment (or any HTTP client) #### Hugging Face Model Access This tutorial requires a Hugging Face token to be configured as an environment variable in your custom application. **Creating Your Hugging Face Token:** 1. Log into your [Hugging Face account](https://huggingface.co/) 2. Navigate to **Settings** → **Access Tokens** 3. Create a new token with **Read** permissions 4. Copy the token value for use in Denvr Dataworks's environment variable configuration **Model License Requirements:** Before deployment, visit the [Mistral-7B model page](https://huggingface.co/mistralai/Mistral-7B-v0.1) and accept any required license agreements. vLLM will not complete initialization if your account hasn't accepted the model's usage terms. ### Create the Deployment Using Denvr Dataworks #### Step 1: Navigate to Application Catalog 1. **Login to Denvr Dataworks**: Access your organization's Denvr Dataworks console 2. **Navigate to Applications**: Go to **Applications** → **Catalog** from the main navigation 3. **Select Custom Application**: Look for the **Custom Application** card 4. **Initialize Configuration**: Click to start the custom application deployment #### Step 2: Basic Information Configuration * **Application Name**: Use a descriptive, systematic naming convention: * Example: `vllm-[model-name]-[environment]-[version]` * **Reserved Resource Pools Only**: Custom applications are currently only allowed through reserved resource pools #### Step 3: Custom Application Configuration In the **Application** section (custom application specific fields): * **Image Repository Type**: Select `Public` (since we're using the official Docker Hub image) * **Container Image URL**: `docker.io/vllm/vllm-openai:v0.10.1.1` #### Step 4: Hardware Configuration In the **Instance Type Configuration** section, select a GPU with sufficient VRAM for your model: **For Mistral-7B (14-16GB VRAM required):** * **Recommended**: A100 MIG `3g.20gb` (20GB) - Cost-effective for 7B models * **Alternative**: A100 Full (40GB) - Higher throughput > **💡 VRAM Estimates**: Requirements assume FP16 inference. Using quantization (4-bit/8-bit) can significantly reduce VRAM needs. #### Step 5: Environment Variables In the **Environment Variables** section, configure the following variables: **Required:** * `HF_TOKEN` - Your Hugging Face User Access Token obtained in the prerequisites **Recommended - Cache Configuration:** To avoid filling up the limited root filesystem storage, configure caching to use direct-attached storage: ```env HF_TOKEN=hf_xxxxxxxxxxxxxxxxxxxxx XDG_CACHE_HOME=/mnt/direct-attached/.cache HF_HOME=/mnt/direct-attached/.cache/huggingface HUGGINGFACE_HUB_CACHE=/mnt/direct-attached/.cache/huggingface/hub CUDA_CACHE_PATH=/mnt/direct-attached/.cache/nv/ComputeCache PIP_CACHE_DIR=/mnt/direct-attached/.cache/pip VLLM_CACHE_ROOT=/mnt/direct-attached/.cache/vllm VLLM_ASSETS_CACHE=/mnt/direct-attached/.cache/vllm/assets ``` > **💡 Cache Benefits**: Configuring cache directories in direct-attached storage prevents filling up the limited root filesystem. Model downloads, compiled CUDA kernels, and other cached data can consume significant space and should be stored on direct-attached storage. #### Step 6: Advanced Settings In the **Advanced Settings** section, configure both the command override and port settings: **Command Override Configuration:** In the **Command Override** field, enter the following command. You can add additional vLLM engine arguments (like `--max-model-len 8192` or `--enable-prefix-caching`) after `--port 8000` and before the `&` symbol to customize performance and behavior. For a complete list of available vLLM engine arguments, see the [vLLM Engine Arguments documentation](https://docs.vllm.ai/en/latest/configuration/engine_args.html). ```bash ["/bin/bash", "-c", "vllm serve mistralai/Mistral-7B-v0.1 --download-dir /mnt/direct-attached/vllm-cache --host 127.0.0.1 --port 8000 & sleep infinity"] ``` > **ℹ️ About Command Override**: This field allows you to specify exactly how the container should start, overriding the default Docker image command. #### Step 7: Proxy Settings In the **Proxy Settings** section, configure the reverse proxy for your deployment: **Port Configuration:** * **Proxy Port**: Set to `8000` (required to enable the reverse proxy and route external traffic to your vLLM service) **API Authentication (Optional but Recommended):** * **API Keys**: Add your API keys in the **API Keys** field to enable secure access through the reverse proxy. > **ℹ️ Proxy Port**: This setting is required to enable the reverse proxy layer. When set, all external traffic routes through the reverse proxy to your vLLM service on this port. > **💡 API Key Best Practices**: Use descriptive prefixes like `sk-prod-`, `sk-staging-`, or `sk-dev-` to identify different keys. Keep these keys secure and never commit them to version control. > **ℹ️ Without API Keys**: If you don't configure any API keys but set the Proxy Port, requests will pass through the reverse proxy without authentication requirements. #### Step 9: Deploy the Application 1. Review all configuration settings 2. Click **Deploy Application** 3. Monitor the deployment status in the **Applications Overview** or **Application Details** 4. To view runtime logs, click on your deployment name and navigate to the **Runtime Logs** tab to monitor the model download and vLLM initialization progress The vLLM initialization process will: * Pull the vLLM Docker image * Start the container with your specified configuration * Download the Mistral-7B model weights from Hugging Face * Initialize the vLLM server with the OpenAI-compatible API > **ℹ️ Deployment Time**: Initial deployment may take 5-10 minutes as the system downloads the \~14GB model weights. The application will show as **ONLINE**, but the vLLM OpenAI-compatible endpoints will not be accessible until initialization is complete. You can monitor progress by viewing the runtime logs in the application details page. ### Alternative: Deploy Using API You can also use the Denvr Dataworks API directly to create Custom Applications. #### Using the Swagger API Interface 1. **Access Swagger UI**: Navigate to 2. **Login**: Log in with your user information (tenancy name, username/email, and password) 3. **Find the Endpoint**: Locate `POST /api/v1/servers/applications/CreateCustomApplication` 4. **Try It Out**: Click "Try it out" to enable the request body editor 5. **Configure Request**: Customize and paste the JSON configuration below into the request body 6. **Execute**: Click "Execute" to create your custom application #### JSON Configuration Template ```json { "name": "vllm-mistral-7b-v1", "cluster": "Msc1", "hardwarePackageName": "g-nvidia-1xa100-20gb-pcie-6vcpu-45gb", "imageUrl": "docker.io/vllm/vllm-openai:v0.10.1.1", "imageCmdOverride": [ "/bin/bash", "-c", "vllm serve mistralai/Mistral-7B-v0.1 --download-dir /mnt/direct-attached/vllm-cache --host 127.0.0.1 --port 8000 & sleep infinity" ], "environmentVariables": { "HF_TOKEN": "hf_xxxxxxxxxxxxxxxxxxxxx", "XDG_CACHE_HOME": "/mnt/direct-attached/.cache", "HF_HOME": "/mnt/direct-attached/.cache/huggingface", "HUGGINGFACE_HUB_CACHE": "/mnt/direct-attached/.cache/huggingface/hub", "CUDA_CACHE_PATH": "/mnt/direct-attached/.cache/nv/ComputeCache", "PIP_CACHE_DIR": "/mnt/direct-attached/.cache/pip", "VLLM_CACHE_ROOT": "/mnt/direct-attached/.cache/vllm", "VLLM_ASSETS_CACHE": "/mnt/direct-attached/.cache/vllm/assets" }, "imageRepository": { "hostname": "https://index.docker.io/v1/", }, "resourcePool": "your-reserved-resource-pool", "proxyPort": 8000, "proxyApiKeys": ["your-api-key1", "your-api-key2s"], "persistDirectAttachedStorage": false, "personalSharedStorage": true, "tenantSharedStorage": true, "securityContext": { "runAsRoot": true, "containerUid": null, "containerGid": null } } ``` **Key Parameters to Customize:** * `name`: Your application name * `cluster`: Your cluster name (e.g., "Msc1") * `hardwarePackageName`: Match available hardware in your cluster * `resourcePool`: Your reserved resource pool name * `HF_TOKEN`: Your Hugging Face token * `proxyApiKeys`: Array of API keys for reverse proxy authentication ### Accessing Your vLLM Deployment #### Finding Your Application's DNS Address 1. Navigate to **Applications** → **Overview** 2. Find your vLLM deployment in the list - the DNS address is displayed in the overview table 3. Alternatively, click on the deployment name to view **Application Details** where you'll also find the DNS address Once you have the DNS address, you can access your vLLM endpoint at that address (e.g., `https://abc-123-456-789-012.cloud.denvrdata.com`) #### Authentication If you configured API keys, your vLLM endpoint will require authentication through the reverse proxy. Include one of your configured API keys in all requests as a bearer token: ```bash -H "Authorization: Bearer your-api-key" ``` Replace `your-api-key` with one of the API keys you set (e.g., `API_KEY_1`, `API_KEY_2`, etc.). > **ℹ️ Without API Keys**: If you didn't configure any API keys but set the Proxy Port, requests will pass through the reverse proxy without authentication requirements. ### Testing Your Deployment #### 1. Health Check First, verify that your vLLM server is running by checking the health endpoint: ```bash curl https://your-dns-address/health \ -H "Authorization: Bearer your-api-key" ``` A successful response returns `HTTP/1.1 200 OK` with no body, indicating the server is healthy and ready to accept requests. #### 2. List Available Models Once the health check passes, verify your model is loaded and accessible: ```bash curl https://your-dns-address/v1/models \ -H "Authorization: Bearer your-api-key" ``` This should return a response showing the `mistralai/Mistral-7B-v0.1` model is available: ```json { "object": "list", "data": [ { "id": "mistralai/Mistral-7B-v0.1", "object": "model", "created": 1737380356, "owned_by": "vllm", "root": "mistralai/Mistral-7B-v0.1", "parent": null, "max_model_len": 32768, "permission": [ { "id": "modelperm-e3d0f87e19a548b2be64ca274a4550a6", "object": "model_permission", "created": 1737380356, "allow_create_engine": false, "allow_sampling": true, "allow_logprobs": true, "allow_search_indices": false, "allow_view": true, "allow_fine_tuning": false, "organization": "*", "group": null, "is_blocking": false } ] } ] } ``` ### Using Your vLLM Deployment #### Using curl (Command Line) The simplest way to test your deployment is with curl commands: **Text Completion** ```bash curl https://your-dns-address/v1/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer your-api-key" \ -d '{ "model": "mistralai/Mistral-7B-v0.1", "prompt": "Explain machine learning in simple terms:", "max_tokens": 150, "temperature": 0.7 }' ``` **Metrics** vLLM exposes Prometheus metrics for monitoring performance and resource utilization: ```bash curl https://your-dns-address/metrics \ -H "Authorization: Bearer your-api-key" ``` This returns metrics like request counts, GPU memory usage, and latency measurements that you can use for monitoring and optimization. > **ℹ️ Note**: If you didn't configure API keys, omit the `-H "Authorization: Bearer your-api-key"` header from all commands. #### Simple Python Client Here's a simple Python client to get you started with your vLLM deployment: ```python import requests def test_vllm_deployment(endpoint_url, api_key): """Simple function to test your vLLM deployment.""" # Test basic completion response = requests.post( f"{endpoint_url}/v1/completions", headers={"Authorization": f"Bearer {api_key}"}, json={ "model": "mistralai/Mistral-7B-v0.1", "prompt": "Explain machine learning in simple terms:", "max_tokens": 100, "temperature": 0.7 } ) if response.status_code == 200: result = response.json() print("Generated text:", result['choices'][0]['text']) return True else: print(f"Error: {response.status_code} - {response.text}") return False # Example usage if __name__ == "__main__": endpoint_url = "https://your-dns-address" # Replace with your actual DNS address api_key = "your-api-key" # Replace with one of your configured API keys test_vllm_deployment(endpoint_url, api_key) ``` > **ℹ️ Note**: If you didn't configure API keys, you can omit the `Authorization` header or pass `None` as the `api_key`. #### Using with OpenAI SDK Your vLLM deployment is compatible with the OpenAI Python SDK: ```python from openai import OpenAI # Initialize client with your vLLM endpoint client = OpenAI( base_url="https://your-dns-address/v1", api_key="your-api-key", # Use one of your configured API keys ) # Generate text completion completion = client.completions.create( model="mistralai/Mistral-7B-v0.1", prompt="Explain the benefits of containerized ML inference:", max_tokens=150, temperature=0.7 ) print(completion.choices[0].text) ``` > **ℹ️ Note**: If you didn't configure API keys, you can set `api_key="not-needed"` or any string value. > > **💡 Base vs Instruction Models**: `mistralai/Mistral-7B-v0.1` is a base model, so we use `client.completions.create()` with a `prompt`. For instruction-tuned models (like `mistralai/Mistral-7B-Instruct-v0.1`), you would use `client.chat.completions.create()` with `messages`. ### Quick Reference #### Essential curl Commands Once your deployment is running, use these commands to interact with your vLLM endpoint. Replace `your-dns-address` with your deployment's DNS address and `your-api-key` with one of your configured API keys: ```bash # Health check curl https://your-dns-address/health \ -H "Authorization: Bearer your-api-key" # List available models curl https://your-dns-address/v1/models \ -H "Authorization: Bearer your-api-key" # Text completion curl https://your-dns-address/v1/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer your-api-key" \ -d '{ "model": "mistralai/Mistral-7B-v0.1", "prompt": "Your prompt here", "max_tokens": 100, "temperature": 0.7 }' ``` #### Common Request Parameters Key parameters for the `/v1/completions` endpoint: * **`prompt`**: The input text to generate completion for (required) * **`max_tokens`**: Maximum tokens to generate (optional, defaults to model's max length) * **`temperature`**: Randomness control, 0.0-2.0 (default: 1.0) * **`stream`**: Enable streaming responses (default: false) See the [vLLM OpenAI-Compatible Server docs](https://docs.vllm.ai/en/latest/serving/openai_compatible_server.html) for all parameters. #### Troubleshooting | Issue | Solution | | ------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------- | | Connection refused | Check the runtime logs to see if vLLM is still initializing | | Unauthorized error | Verify your API key matches one of the configured API\_KEY\_\* seen in the proxy settings | | Model not found | Ensure model name exactly matches: `mistralai/Mistral-7B-v0.1` | | RuntimeError: Engine core initialization failed. | This is likely an out-of-memory error — choose a GPU with more VRAM, use a smaller model or reduce batch size, or enable quantization (e.g., 4-bit) | # Secure Shell (SSH): Best Practices This page aims to shed light on SSH, its importance, and best practices to ensure uninterrupted and secure access to your machines, applications and containers. ### What is SSH? Secure Shell (SSH) is a cryptographic network protocol that provides a secure way to access remote systems over an unsecured network. It is widely used for managing servers, virtual machines, and network devices. SSH ensures confidentiality, integrity, and authentication by encrypting the communication between the client and the server. ### How SSH functions? SSH operates on a client-server model. Here's a quick breakdown: 1. **Client-Server Communication:** A user (client) initiates a connection request to a remote machine (server) running an SSH daemon. 2. **Authentication:** The server authenticates the client using a combination of username and either a password or an SSH key. 3. **Secure Session:** Once authenticated, a secure session is established, allowing the user to execute commands, transfer files, or perform administrative tasks. ### Role of SSH Keys SSH keys are a pair of cryptographic keys (private and public) used for authentication: 1. **Private Key:** Stored securely on your local machine. **Never share this key.** 2. **Public Key:** Uploaded to the `~/.ssh/authorized_keys` file on the server. The server uses this to validate your private key. This method is more secure than password authentication because keys are harder to crack and eliminate the need to transmit sensitive information like passwords over the network. ### Why Permissions are Important? Permissions on your SSH keys and related files are crucial for security. Incorrect permissions can lead to vulnerabilities or authentication failures. • **Private Key:** Must have restricted access (e.g., `chmod 600 ~/.ssh/id_rsa`). • **Public Key:** Can have more permissive access but should still be controlled. • **Authorized Keys File:** Ensure it is properly secured (e.g., `chmod 600 ~/.ssh/authorized_keys`). Improper permissions can result in your keys being accessible to unauthorized users, compromising your access and the security of the VM. ### Common Scenarios Leading to Lost Access: 1. **Lost SSH Key:** Misplacing the private key file. 2. **Changed Permissions:** Altering permissions on `~/.ssh` or `authorized_keys` inadvertently. 3. **Modified Authorized Keys:** Overwriting or accidentally deleting entries in the `authorized_keys` file. ### How to Prevent Loss of Access: 1. **Backup Your Keys:** Always keep a backup of your private key in a secure location. 2. **Use a Password Manager:** Store and manage SSH keys securely using tools like 1Password or LastPass. 3. **Set Correct Permissions:** * Private key: `chmod 600 ~/.ssh/id_rsa` * Public key: `chmod 644 ~/.ssh/id_rsa.pub` * .ssh directory: `chmod 700 ~/.ssh` 4. **Restrict Access:** Limit who can access your private key. Ensure your local machine is secure. 5. **Audit Regularly:** Periodically review and clean up the authorized\_keys file to remove unused or obsolete keys. ### What to Expect in Case of Loss of Access due to SSH Issues? If your SSH key is lost, permissions are incorrect, or the `authorized_keys` file has been modified and no backup or console access is available, you will permanently lose access to your VM or container. This is why it is critical to follow best practices and maintain backups of your keys. If you lose access to your VM due to SSH issues, contact support through support portal available on console. ### Best Practices to Keeping SSH Secure: 1. **Use Key-Based Authentication:** Avoid password-based SSH logins entirely. 2. **Disable Root Login:** Prevent direct root access by disabling it in the SSH configuration file (/etc/ssh/sshd\_config). 3. **Keep SSH Updated:** Regularly update SSH software to patch vulnerabilities. 4. **Use SSH Agent:** Manage your private keys securely in memory using an SSH agent like ssh-agent or gpg-agent. 5. **Rotate Keys Periodically:** Generate new keys periodically and update them on all authorized servers. ### Summary: SSH is the backbone of secure access to your cloud-hosted infrastructure. By understanding how SSH works and following best practices, you can minimize risks, prevent losing access, and maintain seamless connectivity to your VMs. At Denvr, we are committed to empowering users with the tools and knowledge to manage their systems securely and efficiently. # API Usage samples ### Authenticate ``` curl -X 'POST' \ 'https://api.cloud.denvrdata.com/api/TokenAuth/Authenticate' \ -H 'accept: text/plain' \ -H 'Content-Type: application/json-patch+json' \ -d '{ "userNameOrEmailAddress": "YOUR_EMAIL_ADDRESS", "password": "YOUR_PASSWORD" }' ``` Response: ``` { "result": { "accessToken": "eyJhbGci...", "encryptedAccessToken": "l6/HiYvjFp...", "expireInSeconds": 86400, "shouldResetPassword": false, "passwordResetCode": null, "userId": 12345, "requiresTwoFactorVerification": false, "twoFactorAuthProviders": null, "twoFactorRememberClientToken": null, "returnUrl": null, "refreshToken": "eyJhbGciOi...", "refreshTokenExpireInSeconds": 31536000 }, "targetUrl": null, "success": true, "error": null, "unAuthorizedRequest": false, "__abp": true } ``` ### /api/v1/clusters/GetAll ``` curl -X 'GET' \ 'https://api.cloud.denvrdata.com/api/v1/clusters/GetAll' \ -H 'accept: */*' \ -H 'Content-Type: application/json-patch+json' \ -H 'Authorization: Bearer eyJhbGci...' ``` Response: ``` { "result": [ "Hou1","Msc1" ], "targetUrl": null, "success": true, "error": null, "unAuthorizedRequest": false, "__abp": true } ``` ### /api/v1/servers/virtual/CreateServer ``` curl -X 'POST' \ 'https://api.cloud.denvrdata.com/api/v1/servers/virtual/CreateServer' \ -H 'accept: */*' \ -H 'Content-Type: application/json-patch+json' \ -H 'Authorization: Bearer eyJhbGci...' \ -d '{ "name": "my-test-vm", "rpool": "on-demand", "vpc": "YOUR_VPC_NAME", "configuration": "A100_80GB_SXM_1x", "cluster": "Hou1", "ssh_keys": [ "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC..." ], "operatingSystemImage": "Ubuntu 22.04.4 LTS", "rootDiskSize": 500 }' ``` Response: ``` { "result": { "username": "YOUR_EMAIL_ADDRESS", "tenancy_name": "YOUR_TENANCY_NAME", "rpool": "on-demand", "directAttachedStoragePersisted": false, "id": "my-test-vm", "namespace": "YOUR_VPC_NAME", "configuration": "A100_80GB_SXM_1x", "storage": 2125, "gpu_type": "nvidia.com/A100SXM480GB", "gpus": 1, "vcpus": 24, "memory": 116, "ip": "", "privateIp": "172...", "image": "Ubuntu_22.04.4_LTS", "cluster": "Hou1", "status": "na", "storageType": "na" }, "targetUrl": null, "success": true, "error": null, "unAuthorizedRequest": false, "__abp": true } ``` # Registration Step-by-step guide creating a Denvr AI Cloud account To sign in or register for a new account navigate to[ console.cloud.denvrdata.com](https://console.cloud.denvrdata.com).
Hitting the "Register" button will take you to a simple signup form. The company name will be used to name your account/tenant and the administrator information will be used to create the initial admin user. You can always add other users and account admins at a later time. Once you hit the "Submit" button you'll receive confirmation that "you've been successfully registered!"
Once we receive your request we'll setup your account and send an account verification email to the address you provided us. If you don't receive the account verification please check your spam folder for emails coming from .
From this email you can either click on the "Verify" button or copy the provided URL into your browser. Once the email verification is complete you'll be directed to the sign in page shown above. Enter the "Username" and "Temporary Password" provided in the email login. You'll then be prompted to change your password before proceeding to the dashboard.
Accepting the "terms of service" document will conclude the registration process on Denvr AI Cloud. Congratulations!
Checkout our [guide ](/docs/platform/virtual-machines)on bringing up a virtual machine. # Data centers Denvr's' AI clusters are available for use with container workloads, virtual machines, and bare metal hosts. ## Regions AI Services are publicly available in multiple North American locations. * Each region is in a separate geographic area to support data sovereignty requirements * Regions are isolated from other regions with no data replicated between them * Data centers are spread across multiple data halls to ensure power redundancy and availability Denvr operates world-class, highly-available data centers. However AI compute may require unavoidable hardware maintenance due to the high performance nature of data center GPUs and network fabrics. Code development should use multiple nodes and regions to ensure availability and fault tolerance.
Region IdLocation
MSC1Canada West (Calgary)
HOU1US South (Houston)
Denvr maintains compliance with SOC 2 Type 2 and GDPR, and others as applicable. ## GPU Availability The following GPU platforms are available in each public region:
VendorNode TypesGPU FabricHOU1MSC1HOU1MSC1
Intel8x Gaudi 2 96GB2.4 Tbps (RoCE v2)truefalseYes-
NVIDIA 8x H100 80GB SXM3.2 Tbps (IB)truefalseYes-
8x A100 80GB SXM1.6 Tbps (IB)truetrueYesYes
8x A100 40GB SXM800 Gbps (IB)falsetrue-Yes
1x GH200 96GB SXM-falsetrue
4x A100 40GB PCIe-falsetrue-Yes
4x A40 48GB PCIe-falsetrue-Yes
AMD2x Mi210 64GB PCIe-falsetrue
* Nodes are available in bare metal configurations with 8X GPUs and virtual machines with 1X, 2X, 4X and 8X GPUs. * GPU fabrics are non-blocking, rail-optimized fat-tree topology. Bare metal nodes are available in Reserved contracts while VMs are available both On-Demand and Reserved. ## Storage Platforms The following network attached storage platforms are available in each region:
TypePerformance ClassHOU1MSC1
BlockHigh-Performance NVMe - Up to 10 GB/struefalse
FileHigh-Performance NVMe - Up to 10 GB/struefalse
General Purpose - Up to 1 GB/sfalsetrue
## Internet Access The following internet access is available in each region; ISPs are shared by region tenants.
RegionBandwidthRedundant Subscription
HOU1100GYes
MSC110GNo
# Technical support Getting support on Denvr AI Cloud Denvr provides 3 support channels to our clients: 1. [Tickets](#tickets) 2. [Email](#email) 3. [Slack](#slack) ## Tickets Tickets are the preferred method for contacting our technical support team if you already have access to the Denvr AI Cloud web console. Without leaving the dashboard you can: * See the status of open tickets * Review closed tickets * Communicate the severity and impacted services * Upload screenshots Simply navigate to "Support" from the left navigation menu
or from the following links. * [All Tickets](https://console.cloud.denvrdata.com/app/main/support) * [Support Ticket](https://console.cloud.denvrdata.com/app/main/support/new-support-ticket) * [Inquiry](https://console.cloud.denvrdata.com/app/main/support/new-general-inquiry)
## Email In addition to web tickets we can also be reached at `support@denvrdata.com`. Please be sure to include: * Tenant name * Issue priority and impact * Description and screen captures if available Please note that we will validate that the sending email address matches as active user in the Tenant. This is to prevent any impersonation and ensure any access control is followed. We may contact the Tenant Administrator if there are any concerns. ## Slack Enterprises can request access to dedicated Slack channels for chat-based support. # What's new! We're excited to announce updates to the Denvr AI Cloud! Here's what's new: ## Recent updates ### Q3 2025 1. **Bare metal**: web console access to the host for O/S and driver boot issues. 2. **Bare metal**: management UI and APIs to see reserved instances and perform power cycle and O/S redeploy actions. 3. **Account types:** ability to auto-reload balance using stored VISA payment method 4. **Dark mode**: enhancements for support of light and dark mode preferences. ### Q2 2025 1. **Applications:** support for custom apps which are docker workloads dynamically pulled from Dockerhub, github and private container registeries. 2. **API:** we've released v0.1.0 of our [Terraform API](https://registry.terraform.io/providers/denvrdata/denvr/latest). This release supports creating and managing virtual machines. 3. **Inference endpoints:** Beta program available for interested partners to test shared community models as well as dedicated endpoints. ### Q1 2025 1. **Applications**: 1x, 2x, 4x and 8x H100 apps are now available as part of the beta applications service. 2. **API**: we've released v0.1.0 of our Python SDK on [pypi](https://pypi.org/project/denvr/). Our goal is to keep the API minimal while providing a simple config and auth management. The code is available on [github](https://github.com/denvrdata/denvrpy) and feedback is welcome. ### Q4 2024 1. **Account types:** we now support pre-paid accounts with VISA payments via Stripe. Tenant resources will auto-stop at $0 balances along with notification alerts. 2. **Test drives:** new test drives can use either Test credits, or dedicated machines based on customer requirements. 3. **Applications**: early-access release for [Application Catalog](https://docs.denvrdata.com/docs/platform/applications). These are focused on data scientists who prefer a fast-start environment like PyTorch, RAPIDS and JAX along with a JupyterLab interface. 4. **Support tickets**: we have integrated [web support tickets](https://docs.denvrdata.com/docs/overview/technical-support#contact-us-via-the-web) into the console. This new system is easier to use than our previous ticketing platform. 5. **Support Slack**: Denvr Support is available on Slack for tenants with reserved clusters. We also support Teams and Discord if required. ## Past updates ### Q3 2024 1. **Intel Gaudi 2**: general availability of [Intel Gaudi 2](https://www.intel.com/content/www/us/en/products/details/processors/ai-accelerators/gaudi2.html) accelerators in bare metal or VM configurations. Gaudi2 provides superior TCO for both training and inferencing workloads. 2. **NVIDIA H100**: new availability of H100 nodes for on-demand and reserved use including 3,200G InfiniBand ### Q2 2024 1. **Virtual machines:** simplified provisioning to optionally install GPU drivers, mount network-attached storage, and specify root disk size 2. **Bare metal:** test nodes available with [NVIDIA GH200 Grace Hopper Superchip](https://www.nvidia.com/en-us/data-center/grace-hopper-superchip/)! 3. **Docs:** launched new documentation portal [Denvr Docs](https://docs.denvrdata.com/docs) including updated Quick Start guides, feature overviews, API Reference, and FAQs. ### Q1 2024 1. **Virtual machines**: new configurations available including NVIDIA A40, AMD Mi250, AMD cpu-only as well as Windows operating system images. 2. **Slack**: launch of Slack Connect for direct access to Denvr ML support analysts for Bare Metal clusters. # Dashboard The dashboard is our management console to interact with the Denvr AI Compute. From the [dashboard](https://console.cloud.denvrdata.com) you can: * Create and manage GPU resources as * [Container applications](/docs/platform/applications) (including NVIDIA NGC, Jupyter Lab, and user provided) * [Virtual machines](/docs/platform/virtual-machines) * [Bare metal reservations](/docs/platform/bare-metal) * View on-demand and reserved historic utilization * Manage [users and roles](/docs/platform/user-management) * Manage [network VPCs](/docs/platform/networking) # Applications Containerized applications for data scientists and speed of use Applications are docker containers that include all dependencies to run a system. These are provided by hardware vendors or uploaded by the tenant. * [Application Catalog](#application-catalog) * [Creating an Application](#creating-an-application) * [Viewing Applications](#viewing-applications) * [Managing Applications](#managing-applications) * [Managing Storage](#managing-storage) * [Custom Applications](#custom-applications) ## Application Catalog The catalog contains versioned apps that Denvr has selected for various uses. They are focused on data scientists who prefer a fast-start environment like PyTorch, RAPIDS, and JAX along with a JupyterLab. {% hint style="info" %} Denvr can setup new applications as requested including specific versions. {% endhint %}
## Creating an Application Denvr AI Cloud has a single page view to create applications including all of the configuration options that are relevant. ### Basic information This section provides essential details for setting up a container, including the instance name and where resources are allocated from.
NameRefers to the unique identifier or label for the application. It helps users manage and track different containers within their infrastructure.
Resource poolDefines how compute resources are allocated, either on-demand for dynamic allocation or reserved for dedicated single-tenant resources.
### Application This section provides different pre-defined versions of the application to be selected
ReleaseSelect a specific version of the container. This also displays the docker hub source repository used.
### Instance type Defines the overall configuration of the application container, including the type of hardware (such as GPU or CPU), performance characteristics, and resource allocation.
GPU platformRefers to the type of GPU hardware available for running AI and computational tasks. Different platforms offer varying levels of performance and specialization for different workloads.
CPU platformRefers to the type of central processing unit (CPU) hardware available for provisioning. Different CPU platforms offer varying levels of performance for general-purpose computing, suited for tasks that do not require specialized GPU acceleration.
Instance sizeDefines the specific resource allocation, including the number of GPUs, CPUs, memory, and storage, for a virtual machine. It determines the power and capacity of the compute environment.
### Additional storage File volumes can be automatically attached to your machine instances. Volumes can be accessed my multiple instances simultaneously.
PersonalThis is dedicated storage that is only accessible by the specific user, ensuring privacy and security. It is mounted to a unique directory path (e.g., /home/ubuntu/personal).
Tenant sharedThis storage is accessible by multiple users or virtual machines within the same tenant. It is mounted to a shared directory path (e.g., /home/ubuntu/tenant-shared), enabling collaboration and shared access.
### Access credentials Containers support secure access using Jupyter Authentication Tokens for JupyterLab and optional SSH keys for passwordless login. Enable SSH to manage the environment securely with the default user ID and added SSH keys.
Jupyter Authentication TokenApplications that include JupyterLab require a shared password for basic authorization. Please select a difficult password for better security.
SSH KeysSSH (Secure Shell) keys are cryptographic keys used to authenticate access to the application. Users can enter or add additional SSH keys as required.
### Launch Application The right panel shows the application summary. Press `Launch Application` to create the virtual machine.
## Viewing Applications The application detail screen provides an overview of the selected application, including its online status, options to start, stop, or delete, and a direct link to access the JupyterLab interface. Key information about the application, such as instance details, type, hardware configuration, and network settings, is displayed for easy management and monitoring.
## Managing Applications The Applications overview screen displays all running containers and provides fast access to lifecycle actions and access to the JupyterLab interface.
## Managing storage Applications include an ephemeral root filesystem and optional persistent storage volumes. **Root filesystem** The application’s root filesystem (for example, /, /home, and any files written inside the container image) is **ephemeral**. Any changes made to the root filesystem are **permanently deleted** when the application is stopped or deleted. Use the root filesystem for temporary files, package installation during a session, and other non-persistent data only. **Mounted storage** Additional storage volumes are mounted at /mnt inside the application. For catalog applications, convenient symbolic links are also provided under /home/ubuntu. The lifecycle of mounted storage depends on the storage type that is attached. Some storage persists independently of the application, while other storage is deleted with the application. Refer to the [**Storage documentation**](/docs/platform/storage) for details on each storage type. ## Custom Applications > **Note:** This feature is in beta and currently has two restrictions: > > 1. Custom applications can only be deployed on reserved nodes > 2. The image must run a process that keeps the container alive From the application catalog you can also specify a custom application. When launching a custom application you'll need to specify: 1. The registry provider (e.g., docker, github, etc) 2. Whether it is a public or private registry 3. Image URL to pull from 4. Username and password/token for private registries 5. Optional user scripts to be mounted for use in CMD or ENTRYPOINT 6. Optional environment variables to be added 7. Optional proxy port to route https traffic from port 443 8. Optional readiness port to check when your container is online 9. Optional image CMD override
### FAQ A few tips to help you get started with custom applications #### CMD Overrides Inputs should follow the [docker exec form.](https://docs.docker.com/reference/dockerfile/#exec-form) Provide each part of the command as a separate element in the array, unless you explicitly want a shell to interpret the string.\ \ **Invalid** `["jupyter notebook --ip=0.0.0.0"]` **Valid** 1. `["jupyter", "notebook", "--ip=0.0.0.0"]` 2. `["/bin/bash", "-c", "jupyter notebook --ip=0.0.0.0"]` 3. `["/bin/bash", "-c", "echo starting; jupyter notebook && echo ending"]` NOTES: * In the invalid case, the CMD is read as a one argument, which leads to errors. * In all of the valid cases, each argument is passed as a string to a command. * During chaining, `&&` and `;` are used for conditional and unconditional evaluation. See the [bash manual](https://www.gnu.org/savannah-checkouts/gnu/bash/manual/bash.html#Conditional-Constructs) for more details. #### Debugging Container logs do not persist beyond the lifetime of an application. To debug applications we recommend including/starting a remote access service (`sshd`) and then running a command that keeps the applications running. Ex) `["/bin/bash", "-c", "sleep infinity"]` **NOTE** In this case, any commands after `sleep infinity`will not be executed. **Tip** Use a tool like [tini](https://github.com/krallin/tini) to reap zombie process and prevent process table exhaustion. #### Readiness Ports The readiness port just tells Denvr when your application (or virtual machine) is online and accessible. This does not control or restrict what ports are exposed/accessible. See our section on [provisioning states ](/docs/additional-information/faqs/provisioning-states)for more details. # Model endpoints AI Inference on Dedicated GPUs Denvr Model Endpoints provide containers to self-host GPU-accelerated inference services for open source and customized AI models. Model endpoints expose OpenAI-compatible APIs with API Key authentication for fast integration with agentic, chat, and LLM-based systems. * [Model Catalog](#model-catalog) * [Creating an Endpoint](#creating-an-endpoint) * [Managing Endpoints](#managing-model-endpoints) * [Accessing the Endpoint](#accessing-the-endpoint) * [Runtime Logs and Metrics](#runtime-logs-and-metrics) * [Custom Models](#custom-models) ## Model Catalog The model catalog contains the most commonly used GenAI open source models for chat, reasoning, tool calling, and code generation. Each model allows different precision and quantization options to match budget and SLA requirements.\ \ Catalog modes are pre-downloaded to Denvr AI Cloud to increase startup time. {% hint style="info" %} Denvr can enable tenant-specific models or different model parameterization as requested. {% endhint %}
## Creating an Endpoint Configure your model with the following options:
NameUnique identifier or label for the model instance. Allows users to manage and track different models within their infrastructure.
Resource PoolDefines how compute resources are allocated, either on-demand for dynamic allocation or reserved for dedicated single-tenant resources.
ModelSelection of model variants for parameters, active parameters, and model precision
API KeysGenerate or provide secret keys for Bearer-authorization token
Choose the GPU instance to run the model on, and Launch! {% hint style="info" %} Models are validated to work on listed instance types. The optimization engine will use model-parallelism (MP) or data-parallelism (DP) to use all of the GPUs available. {% endhint %} ## Managing Model Endpoints Application overview will display the Model overview, online status, connection info, and access to runtime logs for troubleshooting.
Endpoints can be stopped, restarted, as well as deleted completely. ## Accessing the Endpoint The model overview provides the Private and Public IPs as well as the public DNS. The connection info will show a full example of the HTTPS endpoint and example to use with curl.
The same URL, model, and API KEY can be used in any application or code that is OpenAI-compatible. This includes OpenWebUI, n8n, and OpenCode. ## Runtime Logs and Metrics The application details screen displays the model engine's log file for troubleshooting. {% hint style="info" %} Model runtime logs are currently not available via Denvr API. {% endhint %} ## Custom Models Models not listed in our Catalog, including private models, can be run using our vLLM Server or Ollama Server applications.
The configuration for vLLM Server allows you to specify:
Launch commandOverwrite the container entry command to include vLLM parameters.
Environment variablesView or edit parameters passed into the vLLM engine
# Virtual machines GPU instances for speed and ease of deployment {% hint style="info" %} See FAQ for differences between [Bare Metal and Virtual Machines](/docs/additional-information/faqs/differences-of-bare-metal-and-virtual-machines) {% endhint %} ## Using the Console Denvr AI Compute's console is designed to simplify the provisioning workflow and accelerate time-to-work. It also avoids needing highly technical cloud infrastructure engineers. To view, manage, and deploy VM instances in the Console, navigate to Virtual Machines from the left navigation menu.
Any running virtual machines will be displayed in this screen. This view offers detailed information on a selected machine, as well as actions to start, stop and delete. ## Create a new virtual machine The console has a single page view to create a virtual machine including all of the configuration options that are relevant. ### Basic information This section provides essential details for setting up a virtual machine, including the instance name, where resources are allocated from, and the network isolation for groups of instances.
OptionDescription
NameRefers to the unique identifier or label for the virtual machine (VM). It helps users manage and track different VMs within their infrastructure.
Resource poolDefines how compute resources are allocated, either on-demand for dynamic allocation or reserved for dedicated single-tenant resources.
VPCSpecifies the isolated virtual network in which the VM will operate, allowing users to control network configuration and security for their VM instances.
### Instance type Defines the overall configuration of the virtual machine, including the type of hardware (such as GPU or CPU), performance characteristics, and resource allocation.
OptionDescription
GPU platformRefers to the type of GPU hardware available for running AI and computational tasks. Different platforms offer varying levels of performance and specialization for different workloads.
CPU platformRefers to the type of central processing unit (CPU) hardware available for provisioning. Different CPU platforms offer varying levels of performance for general-purpose computing, suited for tasks that do not require specialized GPU acceleration.
Instance sizeDefines the specific resource allocation, including the number of GPUs, CPUs, memory, and storage, for a virtual machine. It determines the power and capacity of the compute environment.
### Operating system Specifies the software environment that runs on the virtual machine.
OptionDescription
Operating systemUsers can select from different OS options like Ubuntu or Rocky Linux, depending on their application and requirements.
ReleaseDefines the specific version of the operating system.
Machine learning packages

Select whether to install machine learning-related software packages during the creation of the virtual machine.

  • No package install: No additional software will be installed on the instance by default.
  • Install packages: Pre-configured packages such as NVIDIA drivers, Docker, NVIDIA Container Toolkit, NVIDIA Fabric Manager, and InfiniBand drivers will be installed, making the instance ready for machine learning and AI tasks.
Root diskIndicates the storage size allocated to the virtual machine’s main disk, where the operating system and core files will be installed. The root disk is network-attached block storage with replication for redundancy.
### Additional storage File volumes can be automatically attached to your machine instances. Volumes can be accessed my multiple instances simultaneously.
OptionDescription
PersonalThis is dedicated storage that is only accessible by the specific user, ensuring privacy and security. It is mounted to a unique directory path (e.g., /home/ubuntu/personal).
Tenant sharedThis storage is accessible by multiple users or virtual machines within the same tenant. It is mounted to a shared directory path (e.g., /home/ubuntu/tenant-shared), enabling collaboration and shared access.
### Access credentials This section defines the authentication information required to securely access the virtual machine. It includes the default user ID and the option to provide SSH keys for secure, passwordless login.
OptionDescription
User IDThe default username that will be used to log into the virtual machine, such as ubuntu.
SSH KeysSSH (Secure Shell) keys are cryptographic keys used to authenticate access to the virtual machine. Users can enter or add additional SSH keys as required.
## Launch instance The right panel shows the instance summary. Press `Launch instance` to create the virtual machine.
# Bare metal Full access to machine resources without the overhead of virtualization. {% hint style="info" %} See FAQ for differences between [Bare Metal and Virtual Machines](/docs/additional-information/faqs/differences-of-bare-metal-and-virtual-machines) {% endhint %} ## Reserving bare metal hosts Bare metal reservations are processed through our [sales team](https://www.denvrdata.com/contact-our-team) and include sales engineering support to ensure a successful setup. The tenant will need to provide requirements for: * primary SSH public key for remote access * O/S image version or private image * setup of local NVMe as block devices or as file systems (ZFS, RAID level) Denvr can optionally pre-setup: * local NVMe block devices as a RAID filesystem * GPU and InfiniBand drivers (NVIDIA driver, CUDA toolkit, NvLink fabric manager, Mellanox OFED, NCCL) * password-less SSH access between hosts using tenant private network VPC * NFS access to network storage platforms (1PB+ capacity) {% hint style="info" %} Denvr provides CPU-only virtual machine instances for head-node/cluster management software. These will be located in separate chassis and power zones. {% endhint %} ## Accessing bare metal hosts #### SSH Access All hosts can be accessed through their Public IP and SSH key, as well as private VPC networking. Inter-node access uses private networking over 100G ethernet. SSH keys should be configured for password-less authentication for automation purposes. #### Browser-based Console Access > **Note:** This feature is in beta. Contact your administrator to enable console access. All hosts support browser-based console access through a secure web proxy, providing direct terminal access to bare metal systems without requiring SSH client software or inbound internet access (public or VPC). **To access the console:** 1. Navigate to **Bare Metal** from the left navigation bar 2. Find your server in the list and click the **three-dot menu (⋮)** on the right 3. Select **Open Console** from the dropdown menu 4. The browser-based terminal will connect automatically **Console features:** * Full terminal access to OS via IPMI * Copy/paste support for commands and output ## Completing bare metal reservations All host disks will be securely wiped after a reservation period ends. This includes both system disks as well as local NVMe data disks. Please be responsible to backup all data on bare metal hosts to storage platforms, or offsite. {% hint style="warning" %} Host data will be unrecoverable after your reservation completes. {% endhint %} # Storage Denvr AI Compute provide two primary storage categories. * Direct Attached Storage (DAS) * Network Attached Storage (NAS)
## Direct Attached Low latency ephemeral storage, ideal for caching training datasets, model checkpoints and intermediate results. Denvr hosts include up to 30TB with read throughput up to 30 GB/s. On Denvr virtual machines these disks can be formatted and mounted with the following: ```shell-session sudo mkfs -t ext4 /dev/vdc mkdir -p /home/ubuntu/direct-attached mount /dev/vdc /home/ubuntu/direct-attached ``` #### Tips * Cache your models and datasets here to avoid bottlenecks during training or inference tasks. * Backup results or checkpoints to our NAS for reuse on other Denvr hosts. ## Network Attached Reliable distributed network storage for managing VM root disks, saving model checkpoints, and sharing datasets across hosts and even teams. Our network storage comes in **general** and **high-performance** tiers. ### General Storage Low cost redundant storage for home directories, system files, reference datasets or docker images. Petabyte-scale with read throughput up to 1GB/s. #### Tip * Ideal for virtual machine root disks (`/dev/vda`) and custom images. ### High-Performance Storage High performance and redundant storage for sharing large model checkpoints and datasets across a cluster of machines or between users. Petabyte-scale capacity with read throughput up to 10GB/s. #### Tip * Backup DAS cached model checkpoints or training results here for future analysis or retraining. * Personal shares are user specific stores that can be accessed from our virtual machine or application services. Default location is `~/personal`. * Tenant shared can be accessed by any user in an account and is also available from either virtual machines or applications. Default location is `~/tenant-shared`. # Networking ## VPCs Denvr AI Compute supports the creation of multiple VPCs (Virtual Private Cloud) to provide isolation between provisioned resources. {% hint style="info" %} VPCs are local to a region and do not span different Denvr AI Cloud data center regions. {% endhint %} The Console is currently only used to display VPCs. Creating and deleting VPCs is available via the [VPC API.](/docs/api-reference/vpcs)
# User management Manage users and permissions The Tenant Administrator role has the following permissions for user management * Invite others to the team * Remove members from the team * Reset passwords of members * Configure MFA ## Creating users In the dashboard, select Administration - Users then select the button `Create new user`
Enter the name of the user and the email address. On the Roles tab, you can enable two capabilities (default is `User`): | Role | Permissions | | ----------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | TenantAdmin |
  • Apps, VMs, bare metal
  • web support tickets
  • VPC management
  • user management
  • billing management
  • cost and usage
| | User |
  • Apps, VMs, bare metal
  • web support tickets
|
When you select Save, an email will be generated to invite the new user to Tenant. ## Deleting users From the user list, use the Action pull-down button and choose `Delete`.
## Requiring multi-factor (MFA) During user create, or edit create, please check `Two factor authentication enabled`. This will require the user to setup their authenticator on next login.
## Using single sign-on (SSO) Denvr AI Services supports OIDC for single-signon at a tenant level. Please contact the support team to enable this for your organization. # Billing ## On-demand pricing All on-demand instances are billed in one-second increments with pricing quoted per hour. Metering begins when an instance or application in online until it is terminated. {% hint style="info" %} Reserved instances offer significant savings over on-demand due to longer term commitments, pre-payment options, and order size. {% endhint %} {% tabs %} {% tab title="GPU" %}
PlatformInstanceInterconnectPrice/Hour
1x NVIDIA GH20072 vCPU, 480GB RAM, 7.6TB SSDIB 400G$3.87 / GPU
8x H100 80GB SXM208 vCPUs, 1TB RAM, 23TB SSDIB 3200G$2.30 / GPU
8x A100 80GB SXM208 vCPUs, 1TB RAM, 23TB SSD IB 1600G$1.35 / GPU
8x Gaudi2 96GB160 vCPUs, 1TB RAM, 30TB SSDRoCe 2400G$1.25 / GPU
8x A100 40GB SXM128 vCPUs, 1TB RAM, 15TB SSD800B IB$1.15 / GPU
4x A100 40GB PCIe64 vCPUs, 512GB RAM, 7.6TB SSD-$1.15 / GPU
4x A40 48GB PCIe128 vCPU, 512GB RAM, 7.6TB SSD-$0.65 / GPU
Prices are displayed per GPU and include vCPU, system memory and direct-attached NVMe SSD disks. GPU fabrics (InfiniBand and RoCE V2) are available only in full-node instances.\ \ Virtualization configurations are available with 1x, 2x, 4x and 8x GPUs. {% endtab %} {% tab title="CPU" %}
PlatformInstancePrice/Hour
Intel Xeon Gen3 (Ice Lake)8 vCPU, 32GB RAM$0.31
Intel Xeon Gen3 (Ice Lake)16 vCPU, 64GB RAM$0.62
Intel Xeon Gen3 (Ice Lake)32 vCPU, 128GB RAM$1.24
AMD EPYC 3rd Gen (Milan)8 vCPU, 32GB RAM$0.40
AMD EPYC 3rd Gen (Milan)16 vCPU, 64GB RAM$0.79
AMD EPYC 3rd Gen (Milan)32 vCPU, 128GB RAM$1.59
{% endtab %} {% endtabs %} ## Storage pricing File systems are billed per GB used in hourly increments. Storage is billed regardless if compute resources are online.
Storage TypePerformance ClassPrice/GB/mo
BlockHigh-Performance (NVMe SSD)$0.08
File systemsHigh-Performance (NVMe SSD)$0.08
General Purpose$0.03
VM machine imagesHigh-Performance (NVMe SSD)$0.08
SnapshotsHigh-Performance (NVMe SSD)$0.08
## Network pricing Denvr AI Cloud does not currently bill for the following network services: * Internet ingress/egress * Public IPs * VPCs # Billing Administration The Billing Administration page is your central hub for managing credits, reviewing transactions and billing support. This guide walks you through the key features and common tasks. **Access:** Administration → Billing **Required role:** `TenantAdmin` #### Overview The Billing Administration page provides a complete view of your tenancy's financial activity with Denvr Dataworks. Here's what you'll find: * **Tenancy balance** is displayed in the top-right corner and is visible to all users in the tenancy. This balance is shared across the tenancy, and all deployed resources are billed against it. This function is limited to the [prepaid tenants](https://app.gitbook.com/o/HlTERF5RGKb0SX2LPTsH/s/o84Tuz16JeqaVoFstgKj/~/edit/~/changes/298/platform/tenant-types#pay-as-you-go-prepaid). * **Transactions section** lists your full history, including credit top-ups, credit adjustments, promotional credits, and refunds. * **Add Additional Funds** lets `TenantAdmin` users top up credits using a credit card. The default billing currency is USD. This function is limited to the [prepaid tenants](https://app.gitbook.com/o/HlTERF5RGKb0SX2LPTsH/s/o84Tuz16JeqaVoFstgKj/~/edit/~/changes/298/platform/tenant-types#pay-as-you-go-prepaid). * **Export to Excel** allows you to download your transaction history as a `.xlsx` file for your records. * **View Usage** (bottom of the page) opens a detailed breakdown of billed resources. For more information, see [How are my VMs and Applications billed?](https://docs.denvrdata.com/docs/additional-information/faqs/how-are-my-vms-and-applications-billed) * **Contact Support** (bottom of the page) is available for any additional questions. {% hint style="info" %} Denvr Dataworks does not currently support automatic top-ups from a credit card on file. {% endhint %}
#### Adding Credits to Your Tenancy If your tenancy is on the pay-as-you-go billing model, you can top up credits at any time through self-service. **Prerequisites** * The `TenantAdmin` role (required to add credits) * A valid credit card * A minimum top-up amount of **$250 USD** **Payment Processing** Denvr Dataworks has partnered with **Stripe** to provide a secure and seamless payment experience. {% hint style="info" %} Only tenant admins can top up the credits, please reach out to them to add more credits. {% endhint %} #### Invoices: **Access:** Administration → Invoices **Required role:** `TenantAdmin` The Invoices page provides a record of invoices issued to your tenancy. What you see here depends on your billing model. **Who Receives Invoices** * [**Post-paid (standard billed) customers**](https://app.gitbook.com/o/HlTERF5RGKb0SX2LPTsH/s/o84Tuz16JeqaVoFstgKj/~/edit/~/changes/298/platform/tenant-types#standard-billed-postpaid)**:** Invoices are available directly in your dashboard under Administration → Invoices. * [**Pre-paid (pay-as-you-go) customers**](https://app.gitbook.com/o/HlTERF5RGKb0SX2LPTsH/s/o84Tuz16JeqaVoFstgKj/~/edit/~/changes/298/platform/tenant-types#pay-as-you-go-prepaid)**:** Invoices are not generated for manual credit top-ups. Each top-up transaction is recorded in the **Transactions** section of the Billing page, and a payment receipt is issued by Stripe at the time of purchase. **Automatic Top-Ups** Automatic top-ups are available as a feature and disabled for tenancies by default. If you would like to explore enabling automatic top-ups for your tenancy, please reach out through **Contact Support** or **your account manager.** When automatic top-ups are enabled, each automatic credit addition appears as a line item in the **Transactions** section of the Billing page. Invoices for these transactions are not currently issued through the dashboard; receipts are provided directly by Stripe. **Invoice Retention** Invoices are retained in our accounting system and remain accessible through the portal. If you need a copy of a historical invoice that is no longer visible in your dashboard, please contact our support team. #### Adjustments and Refund Requests Before requesting an adjustment, we recommend reviewing **View Usage** for a detailed breakdown of resource consumption across your tenancy. This often clarifies questions about specific charges. If you still have questions or would like to request a credit adjustment or refund, please open a support ticket through **Contact Support** at the bottom of the page, or by selecting **Support** in the left navigation pane. Our accounting team carefully reviews each request, taking into account usage patterns and other relevant factors. We'll work to resolve your request as quickly as possible and will keep you informed throughout the process. ### For our AI Ascend Customers: Thank you for being part of this journey with us. For any billing questions not covered in this guide, please reach out to your account manager directly by email, or raise a ticket through the support portal. # Tenant Types Denvr Dataworks offers three tenant types to suit different usage patterns and customer needs. Your tenant type determines how you're billed, how you add funds, and how invoices are issued. #### **Pay-as-You-Go (Prepaid)** Pay-as-you-go tenancies operate on a prepaid credit model. You add credits to your tenancy in advance, and resource usage is billed against your available balance in real time. * **Billing model:** Prepaid credits * **Adding funds:** Self-service through Administration → Billing → Add Additional Funds * **Minimum top-up:** $250 USD * **Payment method:** Credit card, processed securely through Stripe * **Invoices:** Not issued for credit top-ups; payment receipts are provided by Stripe * **Best suited for:** Customers who prefer flexible, on-demand usage without long-term commitments *** #### **Standard Billed (Postpaid)** Standard billed tenancies operate on a postpaid model. Resources are used throughout the billing cycle, and an invoice is issued for the total usage at the end of the period. * **Billing model:** Postpaid, invoiced periodically * **Adding funds:** Not applicable — usage is invoiced directly * **Invoices:** Issued through the dashboard under Administration → Invoices * **Best suited for:** Customers with predictable, ongoing workloads who prefer consolidated invoicing *** #### **Changing Your Tenant Type** If your needs have changed and you'd like to explore moving between tenant types — for example, from pay-as-you-go to standard billed — please reach out through **Contact Support** or your account manager to discuss available options. # API Keys Long-lived, scoped credentials for API and automation access API keys let you call the Denvr AI Cloud API from scripts, CI/CD pipelines, and applications without embedding your username and password, and without re-authenticating every day. An API key is a long-lived credential issued to **your** user account. It is built on the same token mechanism as the interactive login, with two differences: * **Longer lifetime.** A login access token expires after 24 hours. An API key can be valid for up to a year. * **Scoped permissions.** A login token carries every permission your account has. An API key can be restricted to a subset of them. | | Login access token | API key | | ------------- | ---------------------------------- | ----------------------------------------- | | Obtained from | `POST /api/TokenAuth/Authenticate` | Dashboard → `Administration` → `API Keys` | | Lifetime | 24 hours | Up to 1 year (you choose) | | Permissions | All permissions of your account | All, or a subset you select | | Renewal | Refresh token | Create a new key | | Header | `Authorization: Bearer ` | `Authorization: ApiKey ` | {% hint style="warning" %} An API key is a password-equivalent secret. Anyone who obtains it can act as you, with the permissions the key carries, until it expires or you revoke it. {% endhint %} ### Before you begin * Your account needs the `API Keys` permission. If you do not see `API Keys` under `Administration`, please contact the support team to enable it for your account - see [Technical support.](/docs/overview/getting-started) * Each user may hold up to **10** active API keys at a time. Revoke one before creating another if you reach the limit. * API keys belong to the user who created them. You can only see and revoke your own. ### Creating an API key 1. In the dashboard, go to `Administration` → `API Keys` and select `Create API key`. 2. Enter a **Description**. This is how you will recognize the key later, so name it after where it will be used - for example `ci-nightly-benchmarks` or `terraform-hou1`. 3. Choose an **Expiration**: one week, one month, three months, six months, one year, or a custom date and time. Prefer the shortest period that covers the job - a key that expires is a key that cannot be abused forever. 4. Choose the **Permissions** for the key: * **Use all user permissions** - the key can do everything your account can do. Its effective rights follow your account, so if your role changes later, the key changes with it. * **Use specific permissions** - pick individual permissions from the tree. Selecting a child permission automatically selects its parents. {% hint style="info" %} A key never grants more than its owner has. The effective permissions are the **intersection** of the permissions selected on the key and the permissions your account holds at the time of the request. If an administrator removes a permission from your role, every key you own loses it too. {% endhint %} 5. Select `Create API key`. The key is displayed once. 6. Copy the key and store it in a password manager, a secrets vault, or your CI system's encrypted secret store. Then select `I have saved my API key`. {% hint style="danger" %} Denvr does not store the key and cannot recover it for you - only the last 12 characters are kept, so that you can identify the key in the list. If you lose the key, revoke it and create a new one. {% endhint %} ### Using an API key Send the key in the `Authorization` header using the `ApiKey` scheme: ``` curl -X 'GET' \ 'https://api.cloud.denvrdata.com/api/v1/clusters/GetAll' \ -H 'accept: */*' \ -H 'Content-Type: application/json-patch+json' \ -H 'Authorization: ApiKey eyJhbGciOiJIUzI1NiIsInR5cC...' ``` Response: ``` { "result": [ "Hou1","Msc1" ], "targetUrl": null, "success": true, "error": null, "unAuthorizedRequest": false, "__abp": true } ``` The same header works for write operations: ``` curl -X 'POST' \ 'https://api.cloud.denvrdata.com/api/v1/servers/virtual/CreateServer' \ -H 'accept: */*' \ -H 'Content-Type: application/json-patch+json' \ -H 'Authorization: ApiKey eyJhbGciOiJIUzI1NiIsInR5cC...' \ -d '{ "name": "my-test-vm", "rpool": "on-demand", "vpc": "YOUR_VPC_NAME", "configuration": "A100_80GB_SXM_1x", "cluster": "Hou1", "ssh_keys": [ "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC..." ], "operatingSystemImage": "Ubuntu 22.04.4 LTS", "rootDiskSize": 500 }' ``` In Python, read the key from the environment rather than hard-coding it: ```python import os import requests BASE_URL = "https://api.cloud.denvrdata.com" headers = {"Authorization": f"ApiKey {os.environ['DENVR_API_KEY']}"} response = requests.get(f"{BASE_URL}/api/v1/clusters/GetAll", headers=headers) response.raise_for_status() print(response.json()["result"]) ``` Every endpoint in the API Reference accepts an API key in place of a login token, except `POST /api/TokenAuth/Authenticate` and `POST /api/TokenAuth/RefreshToken`, which exist to issue login tokens and do not need one. ### Reviewing your API keys `Administration` → `API Keys` lists the keys you own: | Column | Meaning | | --------------- | --------------------------------------------------------------- | | Token suffix | The last 12 characters of the key, used to tell your keys apart | | Description | The name you gave the key | | Permissions | The permissions selected for the key, or `All user permissions` | | Expiration date | When the key stops working. Past dates are shown in red | Use `Actions` → `View` to see the full details of a key. The key itself is not shown - only its metadata. ### Revoking an API key Revoke a key as soon as it is no longer needed, and immediately if you suspect it has been exposed - committed to a repository, pasted into a ticket, or written to a build log. 1. Go to `Administration` → `API Keys`. 2. Find the key by its description or token suffix. 3. Select `Actions` → `Revoke`. 4. Type the key's **token suffix** into the confirmation box and select `Revoke`. Revocation takes effect immediately. Any request presenting the revoked key is rejected. Revocation cannot be undone - issue a new key and update your automation. Keys also stop working on their own at the expiration date. Expired keys remain in the list so you can see what existed; revoke them to free up a slot against the 10-key limit. ### Good practice * **Give each consumer its own key.** One key per pipeline, per host, per service. When one is compromised you revoke only that one, and the description tells you what will break. * **Scope the key down.** If a job only reads cluster and VM state, do not give it permission to create servers or manage users. * **Set a short expiration and rotate.** Create the new key, deploy it, verify, then revoke the old one - no downtime. * **Never commit a key.** Use your CI provider's secret store, a secrets manager, or an environment variable loaded at runtime. Add `.env` files to `.gitignore`. * **Never put a key in a URL.** URLs are recorded in browser history, proxy logs, and `Referer` headers. Always use the `Authorization` header. * **Revoke on offboarding.** When someone leaves the team, have them revoke their keys, or delete the user account. ### Troubleshooting | Symptom | Cause | What to do | | ---------------------------------------------------- | ------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `401` or `"unAuthorizedRequest": true` on every call | Header is malformed, or the key has expired or been revoked | Confirm the header reads `Authorization: ApiKey `. The scheme name is case-insensitive, but the space after it is required. Check the key's expiration date in the dashboard. | | `403` on some calls, others succeed | The key is scoped and lacks the permission for that endpoint | Create a new key with the required permission selected, or use `Use all user permissions`. | | A previously working key starts returning `403` | A permission was removed from your account or role | Contact the support team to review the permissions on your account. | | `API Keys` not listed under `Administration` | Your account does not have the `API Keys` permission | Contact the support team to enable it for your account. | | `You have reached the maximum limit of 10 API keys` | The per-user key limit | Revoke a key you no longer use. | If a key has been exposed publicly, revoke it first and then contact Technical support. # Authentication Here, you will find information on endpoints related to token-based authentication, ensuring secure and protected access to our cloud software API. ## POST /api/TokenAuth/Authenticate > Authenticates the user's credentials and, upon successful verification, issues an access token that can be used to authorize subsequent requests made to the API ```json {"openapi":"3.0.4","info":{"title":"DenvrCloud API","version":"v1"},"tags":[{"name":"TokenAuth"}],"paths":{"/api/TokenAuth/Authenticate":{"post":{"tags":["TokenAuth"],"summary":"Authenticates the user's credentials and, upon successful verification, issues an access token that can be used to authorize subsequent requests made to the API","operationId":"ApiTokenauthAuthenticatePost","requestBody":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/AuthenticateModel"}},"text/json":{"schema":{"$ref":"#/components/schemas/AuthenticateModel"}},"application/*+json":{"schema":{"$ref":"#/components/schemas/AuthenticateModel"}}}},"responses":{"200":{"description":"OK","content":{"text/plain":{"schema":{"$ref":"#/components/schemas/AuthenticateResultModel"}},"application/json":{"schema":{"$ref":"#/components/schemas/AuthenticateResultModel"}},"text/json":{"schema":{"$ref":"#/components/schemas/AuthenticateResultModel"}}}}}}}},"components":{"schemas":{"AuthenticateModel":{"required":["password","userNameOrEmailAddress"],"type":"object","properties":{"userNameOrEmailAddress":{"maxLength":256,"minLength":1,"type":"string"},"password":{"maxLength":128,"minLength":1,"type":"string"},"twoFactorVerificationCode":{"type":"string","nullable":true},"rememberClient":{"type":"boolean"},"twoFactorRememberClientToken":{"type":"string","nullable":true},"singleSignIn":{"type":"boolean","nullable":true},"returnUrl":{"type":"string","nullable":true},"captchaResponse":{"type":"string","nullable":true}},"additionalProperties":false},"AuthenticateResultModel":{"type":"object","properties":{"accessToken":{"type":"string","nullable":true},"encryptedAccessToken":{"type":"string","nullable":true},"expireInSeconds":{"type":"integer","format":"int32"},"shouldResetPassword":{"type":"boolean"},"passwordResetCode":{"type":"string","nullable":true},"userId":{"type":"integer","format":"int64"},"requiresTwoFactorVerification":{"type":"boolean"},"twoFactorAuthProviders":{"type":"array","items":{"type":"string"},"nullable":true},"twoFactorRememberClientToken":{"type":"string","nullable":true},"returnUrl":{"type":"string","nullable":true},"refreshToken":{"type":"string","nullable":true},"refreshTokenExpireInSeconds":{"type":"integer","format":"int32"},"c":{"type":"string","nullable":true}},"additionalProperties":false}}}} ``` ## POST /api/TokenAuth/RefreshToken > This endpoint allows for the renewal of the current access token, extending its validity period ```json {"openapi":"3.0.4","info":{"title":"DenvrCloud API","version":"v1"},"tags":[{"name":"TokenAuth"}],"paths":{"/api/TokenAuth/RefreshToken":{"post":{"tags":["TokenAuth"],"summary":"This endpoint allows for the renewal of the current access token, extending its validity period","operationId":"ApiTokenauthRefreshtokenPost","parameters":[{"name":"refreshToken","in":"query","schema":{"type":"string"}}],"responses":{"200":{"description":"OK","content":{"text/plain":{"schema":{"$ref":"#/components/schemas/RefreshTokenResult"}},"application/json":{"schema":{"$ref":"#/components/schemas/RefreshTokenResult"}},"text/json":{"schema":{"$ref":"#/components/schemas/RefreshTokenResult"}}}}}}}},"components":{"schemas":{"RefreshTokenResult":{"type":"object","properties":{"accessToken":{"type":"string","nullable":true},"encryptedAccessToken":{"type":"string","nullable":true},"expireInSeconds":{"type":"integer","format":"int32"}},"additionalProperties":false}}}} ``` ## GET /api/TokenAuth/LogOut > Securely log out a user and invalidate the associated access token, effectively terminating the current session ```json {"openapi":"3.0.4","info":{"title":"DenvrCloud API","version":"v1"},"tags":[{"name":"TokenAuth"}],"paths":{"/api/TokenAuth/LogOut":{"get":{"tags":["TokenAuth"],"summary":"Securely log out a user and invalidate the associated access token, effectively terminating the current session","operationId":"ApiTokenauthLogoutGet","responses":{"200":{"description":"OK"}}}}}} ``` # Clusters On this page, you will find detailed documentation on endpoints related to accessing and managing clusters, enabling efficient scaling and operation within our cloud software API. ## GET /api/v1/clusters/GetAll > Get a list of allocated clusters ```json {"openapi":"3.0.4","info":{"title":"DenvrCloud API","version":"v1"},"tags":[{"name":"clusters"}],"paths":{"/api/v1/clusters/GetAll":{"get":{"tags":["clusters"],"summary":"Get a list of allocated clusters","operationId":"ApiV1ClustersGetallGet","responses":{"200":{"description":"OK","content":{"text/plain":{"schema":{"type":"array","items":{"type":"string"}}},"application/json":{"schema":{"type":"array","items":{"type":"string"}}},"text/json":{"schema":{"type":"array","items":{"type":"string"}}}}},"400":{"description":"Bad Request"},"401":{"description":"Unauthorized"},"403":{"description":"Forbidden"},"500":{"description":"Internal Server Error"}}}}}} ``` # Applications Explore this section for comprehensive information on endpoints related to the deployment, management, and operation of applications within our cloud software API environment. ## GET /api/v1/servers/applications/GetApplications > Get a list of applications ```json {"openapi":"3.0.4","info":{"title":"DenvrCloud API","version":"v1"},"tags":[{"name":"servers/applications"}],"paths":{"/api/v1/servers/applications/GetApplications":{"get":{"tags":["servers/applications"],"summary":"Get a list of applications","operationId":"ApiV1ServersApplicationsGetapplicationsGet","responses":{"200":{"description":"OK","content":{"text/plain":{"schema":{"$ref":"#/components/schemas/ListResultDtoOfApplicationsApiOverview"}},"application/json":{"schema":{"$ref":"#/components/schemas/ListResultDtoOfApplicationsApiOverview"}},"text/json":{"schema":{"$ref":"#/components/schemas/ListResultDtoOfApplicationsApiOverview"}}}},"400":{"description":"Bad Request"},"401":{"description":"Unauthorized"},"403":{"description":"Forbidden"},"500":{"description":"Internal Server Error"}}}}},"components":{"schemas":{"ListResultDtoOfApplicationsApiOverview":{"type":"object","properties":{"items":{"type":"array","items":{"$ref":"#/components/schemas/ApplicationsApiOverview"},"nullable":true}},"additionalProperties":false},"ApplicationsApiOverview":{"type":"object","properties":{"id":{"type":"string","nullable":true},"cluster":{"type":"string","nullable":true},"status":{"type":"string","nullable":true},"tenant":{"type":"string","nullable":true},"createdBy":{"type":"string","nullable":true},"privateIp":{"type":"string","nullable":true},"publicIp":{"type":"string","nullable":true},"resourcePool":{"type":"string","nullable":true},"dns":{"type":"string","nullable":true},"sshUsername":{"type":"string","nullable":true},"applicationCatalogItemName":{"type":"string","nullable":true},"applicationCatalogItemVersionName":{"type":"string","nullable":true},"hardwarePackageName":{"type":"string","nullable":true},"persistedDirectAttachedStorage":{"type":"boolean"},"personalSharedStorage":{"type":"boolean"},"tenantSharedStorage":{"type":"boolean"}},"additionalProperties":false}}}} ``` ## GET /api/v1/servers/applications/GetApplicationDetails > Get detailed information about a specific application ```json {"openapi":"3.0.4","info":{"title":"DenvrCloud API","version":"v1"},"tags":[{"name":"servers/applications"}],"paths":{"/api/v1/servers/applications/GetApplicationDetails":{"get":{"tags":["servers/applications"],"summary":"Get detailed information about a specific application","operationId":"ApiV1ServersApplicationsGetapplicationdetailsGet","parameters":[{"name":"Id","in":"query","description":"The application name","required":true,"schema":{"type":"string"}},{"name":"Cluster","in":"query","description":"The cluster you're operating on","required":true,"schema":{"type":"string"}}],"responses":{"200":{"description":"OK","content":{"text/plain":{"schema":{"$ref":"#/components/schemas/ApplicationsApiDetails"}},"application/json":{"schema":{"$ref":"#/components/schemas/ApplicationsApiDetails"}},"text/json":{"schema":{"$ref":"#/components/schemas/ApplicationsApiDetails"}}}},"400":{"description":"Bad Request"},"401":{"description":"Unauthorized"},"403":{"description":"Forbidden"},"404":{"description":"Not Found"},"500":{"description":"Internal Server Error"}}}}},"components":{"schemas":{"ApplicationsApiDetails":{"type":"object","properties":{"instanceDetails":{"$ref":"#/components/schemas/InstanceDetails"},"applicationCatalogItem":{"$ref":"#/components/schemas/ApplicationsApiCatalogItem"},"hardwarePackage":{"$ref":"#/components/schemas/ApplicationsApiApplicationConfig"},"servingDetails":{"$ref":"#/components/schemas/ApplicationsApiServingDetails"}},"additionalProperties":false},"InstanceDetails":{"type":"object","properties":{"id":{"type":"string","nullable":true},"cluster":{"type":"string","nullable":true},"status":{"type":"string","nullable":true},"statusReason":{"type":"string","nullable":true},"statusMessage":{"type":"string","nullable":true},"privateIp":{"type":"string","nullable":true},"publicIp":{"type":"string","nullable":true},"imageCmdOverride":{"type":"string","nullable":true},"environmentVariables":{"type":"object","additionalProperties":{"type":"string","nullable":true},"description":"The environment variables the application container is running with. Reflects the effective\nconfiguration: variables supplied at creation, values injected from dedicated request fields\n(e.g. jupyterToken), and catalog defaults merged at deployment. Platform-internal variables\n(e.g. SSH key material) are excluded. Values are not redacted and may include secrets, so\ntreat the response as sensitive.","nullable":true},"readinessWatcherPort":{"type":"integer","format":"int32","nullable":true},"proxyPort":{"type":"integer","format":"int32","nullable":true},"createdBy":{"type":"string","nullable":true},"tenant":{"type":"string","nullable":true},"resourcePool":{"type":"string","nullable":true},"nodeSelector":{"type":"string","nullable":true},"creationTime":{"type":"string","format":"date-time"},"lastUpdated":{"type":"string","format":"date-time","nullable":true},"dns":{"type":"string","nullable":true},"persistedDirectAttachedStorage":{"type":"boolean"},"personalSharedStorage":{"type":"boolean"},"tenantSharedStorage":{"type":"boolean"},"runAsRoot":{"type":"boolean","description":"Run container with root privileges. When disabled, requires UID and GID.","nullable":true},"containerUid":{"type":"integer","description":"User ID (UID) for running container when not using root privileges","format":"int32","nullable":true},"containerGid":{"type":"integer","description":"Group ID (GID) for running container when not using root privileges","format":"int32","nullable":true}},"additionalProperties":false},"ApplicationsApiCatalogItem":{"type":"object","properties":{"name":{"type":"string","nullable":true},"applicationSourceDetailsUrl":{"type":"string","nullable":true},"applicationSourceOwner":{"type":"string","nullable":true},"type":{"type":"string","nullable":true},"versions":{"type":"array","items":{"$ref":"#/components/schemas/ApplicationsApiCatalogItemVersion"},"nullable":true}},"additionalProperties":false},"ApplicationsApiCatalogItemVersion":{"type":"object","properties":{"name":{"type":"string","nullable":true},"imageUrl":{"type":"string","nullable":true},"imageLastPushDate":{"type":"string","format":"date-time","nullable":true},"platform":{"type":"string","nullable":true},"launchType":{"type":"string","nullable":true},"releaseNotesUrl":{"type":"string","nullable":true},"accelerator":{"type":"string","nullable":true},"minVramGb":{"type":"integer","format":"int32","nullable":true},"defaultProxyPort":{"type":"string","nullable":true},"defaultEnvironmentVariables":{"type":"object","additionalProperties":{"type":"string","nullable":true},"nullable":true}},"additionalProperties":false},"ApplicationsApiApplicationConfig":{"type":"object","properties":{"name":{"type":"string","nullable":true},"description":{"type":"string","nullable":true},"gpuCount":{"type":"integer","format":"int32","nullable":true},"gpuType":{"type":"string","nullable":true},"gpuBrand":{"type":"string","nullable":true},"gpuName":{"type":"string","nullable":true},"vcpusCount":{"type":"integer","format":"int32","nullable":true},"memoryGb":{"type":"integer","format":"int64","nullable":true},"directAttachedStorageGb":{"type":"integer","format":"int32","nullable":true},"pricePerHour":{"type":"number","format":"double","nullable":true},"vramGb":{"type":"integer","format":"int32","nullable":true},"clusters":{"type":"array","items":{"type":"string"},"nullable":true}},"additionalProperties":false},"ApplicationsApiServingDetails":{"type":"object","properties":{"platform":{"type":"string","nullable":true},"modelId":{"type":"string","nullable":true},"baseUrl":{"type":"string","nullable":true},"healthUrl":{"type":"string","nullable":true},"requiresAuthentication":{"type":"boolean"},"authScheme":{"type":"string","nullable":true}},"additionalProperties":false}}}} ``` ## GET /api/v1/servers/applications/GetConfigurations > Get a list of application configurations ```json {"openapi":"3.0.4","info":{"title":"DenvrCloud API","version":"v1"},"tags":[{"name":"servers/applications"}],"paths":{"/api/v1/servers/applications/GetConfigurations":{"get":{"tags":["servers/applications"],"summary":"Get a list of application configurations","operationId":"ApiV1ServersApplicationsGetconfigurationsGet","responses":{"200":{"description":"OK","content":{"text/plain":{"schema":{"$ref":"#/components/schemas/ListResultDtoOfApplicationsApiApplicationConfig"}},"application/json":{"schema":{"$ref":"#/components/schemas/ListResultDtoOfApplicationsApiApplicationConfig"}},"text/json":{"schema":{"$ref":"#/components/schemas/ListResultDtoOfApplicationsApiApplicationConfig"}}}},"400":{"description":"Bad Request"},"401":{"description":"Unauthorized"},"403":{"description":"Forbidden"},"404":{"description":"Not Found"},"500":{"description":"Internal Server Error"}}}}},"components":{"schemas":{"ListResultDtoOfApplicationsApiApplicationConfig":{"type":"object","properties":{"items":{"type":"array","items":{"$ref":"#/components/schemas/ApplicationsApiApplicationConfig"},"nullable":true}},"additionalProperties":false},"ApplicationsApiApplicationConfig":{"type":"object","properties":{"name":{"type":"string","nullable":true},"description":{"type":"string","nullable":true},"gpuCount":{"type":"integer","format":"int32","nullable":true},"gpuType":{"type":"string","nullable":true},"gpuBrand":{"type":"string","nullable":true},"gpuName":{"type":"string","nullable":true},"vcpusCount":{"type":"integer","format":"int32","nullable":true},"memoryGb":{"type":"integer","format":"int64","nullable":true},"directAttachedStorageGb":{"type":"integer","format":"int32","nullable":true},"pricePerHour":{"type":"number","format":"double","nullable":true},"vramGb":{"type":"integer","format":"int32","nullable":true},"clusters":{"type":"array","items":{"type":"string"},"nullable":true}},"additionalProperties":false}}}} ``` ## GET /api/v1/servers/applications/GetAvailability > Get detailed information on available configurations for applications ```json {"openapi":"3.0.4","info":{"title":"DenvrCloud API","version":"v1"},"tags":[{"name":"servers/applications"}],"paths":{"/api/v1/servers/applications/GetAvailability":{"get":{"tags":["servers/applications"],"summary":"Get detailed information on available configurations for applications","operationId":"ApiV1ServersApplicationsGetavailabilityGet","parameters":[{"name":"cluster","in":"query","description":"","required":true,"schema":{"type":"string"}},{"name":"resourcePool","in":"query","description":"","required":true,"schema":{"type":"string"}},{"name":"applicationCatalogItemName","in":"query","schema":{"type":"string"}},{"name":"applicationCatalogItemVersion","in":"query","schema":{"type":"string"}}],"responses":{"200":{"description":"OK","content":{"text/plain":{"schema":{"$ref":"#/components/schemas/ListResultDtoOfApplicationsApiApplicationConfigAvailability"}},"application/json":{"schema":{"$ref":"#/components/schemas/ListResultDtoOfApplicationsApiApplicationConfigAvailability"}},"text/json":{"schema":{"$ref":"#/components/schemas/ListResultDtoOfApplicationsApiApplicationConfigAvailability"}}}},"400":{"description":"Bad Request"},"401":{"description":"Unauthorized"},"403":{"description":"Forbidden"},"404":{"description":"Not Found"},"500":{"description":"Internal Server Error"}}}}},"components":{"schemas":{"ListResultDtoOfApplicationsApiApplicationConfigAvailability":{"type":"object","properties":{"items":{"type":"array","items":{"$ref":"#/components/schemas/ApplicationsApiApplicationConfigAvailability"},"nullable":true}},"additionalProperties":false},"ApplicationsApiApplicationConfigAvailability":{"type":"object","properties":{"configuration":{"type":"string","nullable":true},"cluster":{"type":"string","nullable":true},"rpool":{"type":"string","nullable":true},"price":{"type":"number","format":"double","nullable":true},"available":{"type":"boolean","nullable":true},"count":{"type":"integer","format":"int32","nullable":true},"maxCount":{"type":"integer","format":"int32","nullable":true},"availableNodeNames":{"type":"array","items":{"type":"string"},"nullable":true}},"additionalProperties":false}}}} ``` ## GET /api/v1/servers/applications/GetApplicationCatalogItems > Get a list of application catalog items ```json {"openapi":"3.0.4","info":{"title":"DenvrCloud API","version":"v1"},"tags":[{"name":"servers/applications"}],"paths":{"/api/v1/servers/applications/GetApplicationCatalogItems":{"get":{"tags":["servers/applications"],"summary":"Get a list of application catalog items","operationId":"ApiV1ServersApplicationsGetapplicationcatalogitemsGet","parameters":[{"name":"types","in":"query","schema":{"type":"array","items":{"type":"string"}}}],"responses":{"200":{"description":"OK","content":{"text/plain":{"schema":{"$ref":"#/components/schemas/ListResultDtoOfApplicationsApiCatalogItem"}},"application/json":{"schema":{"$ref":"#/components/schemas/ListResultDtoOfApplicationsApiCatalogItem"}},"text/json":{"schema":{"$ref":"#/components/schemas/ListResultDtoOfApplicationsApiCatalogItem"}}}},"400":{"description":"Bad Request"},"401":{"description":"Unauthorized"},"403":{"description":"Forbidden"},"500":{"description":"Internal Server Error"}}}}},"components":{"schemas":{"ListResultDtoOfApplicationsApiCatalogItem":{"type":"object","properties":{"items":{"type":"array","items":{"$ref":"#/components/schemas/ApplicationsApiCatalogItem"},"nullable":true}},"additionalProperties":false},"ApplicationsApiCatalogItem":{"type":"object","properties":{"name":{"type":"string","nullable":true},"applicationSourceDetailsUrl":{"type":"string","nullable":true},"applicationSourceOwner":{"type":"string","nullable":true},"type":{"type":"string","nullable":true},"versions":{"type":"array","items":{"$ref":"#/components/schemas/ApplicationsApiCatalogItemVersion"},"nullable":true}},"additionalProperties":false},"ApplicationsApiCatalogItemVersion":{"type":"object","properties":{"name":{"type":"string","nullable":true},"imageUrl":{"type":"string","nullable":true},"imageLastPushDate":{"type":"string","format":"date-time","nullable":true},"platform":{"type":"string","nullable":true},"launchType":{"type":"string","nullable":true},"releaseNotesUrl":{"type":"string","nullable":true},"accelerator":{"type":"string","nullable":true},"minVramGb":{"type":"integer","format":"int32","nullable":true},"defaultProxyPort":{"type":"string","nullable":true},"defaultEnvironmentVariables":{"type":"object","additionalProperties":{"type":"string","nullable":true},"nullable":true}},"additionalProperties":false}}}} ``` ## Create a new application using a pre-defined configuration and application catalog item > Request body examples for common catalog applications (Jupyter Notebook, vLLM Server,\ > Ollama Server, and model applications) are available from the examples dropdown in the\ > request body section below, along with per-application notes.\ > \ > The catalog item's default environment variables are applied automatically; any variable\ > you provide in \*\*environmentVariables\*\* takes precedence over the default. Each version's\ > defaults are returned as \*\*defaultEnvironmentVariables\*\* by the GetApplicationCatalogItems endpoint. ```json {"openapi":"3.0.4","info":{"title":"DenvrCloud API","version":"v1"},"tags":[{"name":"servers/applications"}],"paths":{"/api/v1/servers/applications/CreateCatalogApplication":{"post":{"tags":["servers/applications"],"summary":"Create a new application using a pre-defined configuration and application catalog item","description":"Request body examples for common catalog applications (Jupyter Notebook, vLLM Server,\nOllama Server, and model applications) are available from the examples dropdown in the\nrequest body section below, along with per-application notes.\n \nThe catalog item's default environment variables are applied automatically; any variable\nyou provide in **environmentVariables** takes precedence over the default. Each version's\ndefaults are returned as **defaultEnvironmentVariables** by the GetApplicationCatalogItems endpoint.","operationId":"ApiV1ServersApplicationsCreatecatalogapplicationPost","requestBody":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/ApplicationsApiCreateRequest"}},"text/json":{"schema":{"$ref":"#/components/schemas/ApplicationsApiCreateRequest"}},"application/*+json":{"schema":{"$ref":"#/components/schemas/ApplicationsApiCreateRequest"}}}},"responses":{"202":{"description":"Accepted","content":{"text/plain":{"schema":{"$ref":"#/components/schemas/ApplicationsApiOverview"}},"application/json":{"schema":{"$ref":"#/components/schemas/ApplicationsApiOverview"}},"text/json":{"schema":{"$ref":"#/components/schemas/ApplicationsApiOverview"}}}},"400":{"description":"Bad Request"},"401":{"description":"Unauthorized"},"402":{"description":"Payment Required"},"403":{"description":"Forbidden"},"404":{"description":"Not Found"},"409":{"description":"Conflict"},"422":{"description":"Unprocessable Content"},"500":{"description":"Internal Server Error"}}}}},"components":{"schemas":{"ApplicationsApiCreateRequest":{"required":["applicationCatalogItemName","applicationCatalogItemVersion","cluster","hardwarePackageName","name"],"type":"object","properties":{"name":{"maxLength":47,"minLength":0,"type":"string","description":"The application name.\nMust be 47 characters or less."},"cluster":{"minLength":1,"type":"string","description":"The cluster you're operating on"},"hardwarePackageName":{"minLength":1,"type":"string","description":"The name or unique identifier of the application hardware configuration to use for the application."},"applicationCatalogItemName":{"minLength":1,"type":"string","description":"The name of the application catalog item."},"applicationCatalogItemVersion":{"minLength":1,"type":"string","description":"The version name of the application catalog item."},"resourcePool":{"type":"string","description":"The resource pool to use for the application","nullable":true},"sshKeys":{"type":"array","items":{"type":"string"},"description":"The SSH keys for accessing the application","nullable":true},"persistDirectAttachedStorage":{"type":"boolean","description":"Indicates whether to persist direct attached storage (if resource pool is reserved)"},"personalSharedStorage":{"type":"boolean","description":"Enable personal shared storage for the application"},"tenantSharedStorage":{"type":"boolean","description":"Enable tenant shared storage for the application"},"selectedNode":{"type":"string","description":"Specific node name to target for application deployment.\nUsed for non-on-demand resource pools to allow node-specific scheduling.","nullable":true},"jupyterToken":{"type":"string","description":"An authentication token for accessing Jupyter Notebook enabled applications","nullable":true},"startupCommands":{"type":"array","items":{"type":"string"},"description":"List of startup commands to be executed during container initialization.\nCommands are executed in order and joined with semicolons.\nUsed for custom initialization logic before the main application starts.","nullable":true},"environmentVariables":{"type":"object","additionalProperties":{"type":"string","nullable":true},"description":"Custom environment variables for the application.\nKey-value pairs that will be set in the container environment.\nUseful for authentication tokens, cache paths, and runtime configuration.","nullable":true},"proxyPort":{"type":"string","description":"The port number for the application proxy service. Required to setup the proxy\nUsed in conjunction with proxyApiKeys for authenticated access.","nullable":true},"proxyApiKeys":{"type":"array","items":{"type":"string"},"description":"Optional API keys for authenticating with the application proxy service.\nMultiple keys can be provided to support key rotation.\nEach key must contain only alphanumeric characters, hyphens, and underscores.","nullable":true},"inlineVolumes":{"type":"array","items":{"$ref":"#/components/schemas/InlineVolumeDto"},"description":"Persistent CephFS-backed volumes to provision and mount into the container.\nCurrently only Open WebUI applications use this; capped at one volume per app.","nullable":true},"openWebuiAdminEmail":{"maxLength":254,"minLength":0,"type":"string","description":"Admin user email for Open WebUI initial setup. Required when creating an open-webui application.","format":"email","nullable":true},"openWebuiAdminPassword":{"maxLength":128,"minLength":8,"type":"string","description":"Admin user password for Open WebUI initial setup. Required when creating an open-webui application.\nMinimum 8 characters.","nullable":true}},"additionalProperties":false},"InlineVolumeDto":{"type":"object","properties":{"name":{"type":"string","nullable":true},"capacity":{"type":"string","nullable":true},"mountPoint":{"type":"string","nullable":true}},"additionalProperties":false},"ApplicationsApiOverview":{"type":"object","properties":{"id":{"type":"string","nullable":true},"cluster":{"type":"string","nullable":true},"status":{"type":"string","nullable":true},"tenant":{"type":"string","nullable":true},"createdBy":{"type":"string","nullable":true},"privateIp":{"type":"string","nullable":true},"publicIp":{"type":"string","nullable":true},"resourcePool":{"type":"string","nullable":true},"dns":{"type":"string","nullable":true},"sshUsername":{"type":"string","nullable":true},"applicationCatalogItemName":{"type":"string","nullable":true},"applicationCatalogItemVersionName":{"type":"string","nullable":true},"hardwarePackageName":{"type":"string","nullable":true},"persistedDirectAttachedStorage":{"type":"boolean"},"personalSharedStorage":{"type":"boolean"},"tenantSharedStorage":{"type":"boolean"}},"additionalProperties":false}}}} ``` ## Create a new custom application using a pre-defined configuration and user-defined container image. > Custom applications must be deployed to a reserved resource pool; the on-demand pool\ > is not supported for custom images.\ > \ > Request body examples covering a public container image, a vLLM inference server\ > exposed through the reverse proxy, a private registry, and persistent direct attached\ > storage are available from the examples dropdown in the request body section below. ```json {"openapi":"3.0.4","info":{"title":"DenvrCloud API","version":"v1"},"tags":[{"name":"servers/applications"}],"paths":{"/api/v1/servers/applications/CreateCustomApplication":{"post":{"tags":["servers/applications"],"summary":"Create a new custom application using a pre-defined configuration and user-defined container image.","description":"Custom applications must be deployed to a reserved resource pool; the on-demand pool\nis not supported for custom images.\n \nRequest body examples covering a public container image, a vLLM inference server\nexposed through the reverse proxy, a private registry, and persistent direct attached\nstorage are available from the examples dropdown in the request body section below.","operationId":"ApiV1ServersApplicationsCreatecustomapplicationPost","requestBody":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/ApplicationsApiCustomApiCreateRequest"}},"text/json":{"schema":{"$ref":"#/components/schemas/ApplicationsApiCustomApiCreateRequest"}},"application/*+json":{"schema":{"$ref":"#/components/schemas/ApplicationsApiCustomApiCreateRequest"}}}},"responses":{"202":{"description":"Accepted","content":{"text/plain":{"schema":{"$ref":"#/components/schemas/ApplicationsApiOverview"}},"application/json":{"schema":{"$ref":"#/components/schemas/ApplicationsApiOverview"}},"text/json":{"schema":{"$ref":"#/components/schemas/ApplicationsApiOverview"}}}},"400":{"description":"Bad Request"},"401":{"description":"Unauthorized"},"402":{"description":"Payment Required"},"403":{"description":"Forbidden"},"404":{"description":"Not Found"},"409":{"description":"Conflict"},"422":{"description":"Unprocessable Content"},"500":{"description":"Internal Server Error"}}}}},"components":{"schemas":{"ApplicationsApiCustomApiCreateRequest":{"required":["cluster","hardwarePackageName","imageUrl","name"],"type":"object","properties":{"name":{"maxLength":47,"minLength":0,"type":"string","description":"The application name.\nMust be 47 characters or less."},"cluster":{"minLength":1,"type":"string","description":"The cluster you're operating on"},"hardwarePackageName":{"minLength":1,"type":"string","description":"The name or unique identifier of the application hardware configuration to use for the application."},"imageUrl":{"minLength":1,"type":"string","description":"Image URL for the custom application."},"imageCmdOverride":{"type":"array","items":{"type":"string"},"description":"Optional Image CMD override allows users to specify a custom command to run in the container.\nMust be a JSON array (e.g., [\"python\", \"train.py\"])","nullable":true},"environmentVariables":{"type":"object","additionalProperties":{"type":"string","nullable":true},"description":"Environment variables for the application.\nNames must start with a letter or underscore and contain only alphanumeric characters and underscores.\nValues must not contain null characters, carriage returns, or newlines.","nullable":true},"imageRepository":{"$ref":"#/components/schemas/ImageRepositoryDto"},"resourcePool":{"type":"string","description":"The resource pool to use for the application.\nCustom applications support reserved resource pools only; on-demand is not supported.","nullable":true},"readinessWatcherPort":{"type":"integer","description":"The port used for monitoring application readiness and status.\nCommon examples: \n- 443 (JupyterLab) \n- 22 (SSH)","format":"int32","nullable":true},"proxyPort":{"type":"integer","description":"The port your application uses to receive HTTPS traffic.\nWhen set, a reverse proxy will be automatically configured in front of your application.\nPort 443 is reserved for the reverse proxy and cannot be used.","format":"int32","nullable":true},"proxyApiKeys":{"type":"array","items":{"type":"string"},"description":"API keys for authenticating with the reverse proxy service.\nOptional, but requires proxyPort to be set for the reverse proxy to be configured.\nEach key must:\n- Contain only alphanumeric characters, hyphens, and underscores\n- Not exceed 512 characters\n- Not be null or whitespace\nMaximum 10 keys allowed.","nullable":true},"persistDirectAttachedStorage":{"type":"boolean","description":"Indicates whether to persist direct attached storage (if resource pool is reserved)"},"personalSharedStorage":{"type":"boolean","description":"Enable personal shared storage for the application"},"tenantSharedStorage":{"type":"boolean","description":"Enable tenant shared storage for the application"},"selectedNode":{"type":"string","description":"Specific node name to target for application deployment.\nUsed for non-on-demand resource pools to allow node-specific scheduling.","nullable":true},"userScripts":{"type":"object","additionalProperties":{"type":"string","nullable":true},"description":"Dictionary of script filenames to script content. Each scripts to be mounted at /etc/script/user-scripts for use in CMD or ENTRYPOINT.\nScript names must:\n- Contain only alphanumeric characters, dots, spaces and parentheses\n- Not exceed 255 characters\nScript content must:\n- Not be null or empty\n- Not exceed 16384 characters\n- Not contain invalid characters","nullable":true},"securityContext":{"$ref":"#/components/schemas/SecurityContextDto"}},"additionalProperties":false},"ImageRepositoryDto":{"type":"object","properties":{"hostname":{"type":"string","description":"The registry hostname for the container repository.\nIf not provided in the ImageRepository object, will be inferred from the imageUrl.\nExamples:\n- Docker Hub: \"https://index.docker.io/v1/\"\n- GitHub Container Registry: \"https://ghcr.io/\"","nullable":true},"username":{"type":"string","description":"The username for authentication with private repositories.\nThis is only required if the repository is private.","nullable":true},"password":{"type":"string","description":"The password or access token for authentication with private repositories.\nThis is only required if the repository is private.","nullable":true}},"additionalProperties":false},"SecurityContextDto":{"type":"object","properties":{"runAsRoot":{"type":"boolean","description":"Run container with root privileges. When disabled, requires UID and GID."},"containerUid":{"maximum":2147483647,"minimum":0,"type":"integer","description":"User ID (UID) for running container when not using root privileges","format":"int32","nullable":true},"containerGid":{"maximum":2147483647,"minimum":0,"type":"integer","description":"Group ID (GID) for running container when not using root privileges","format":"int32","nullable":true}},"additionalProperties":false},"ApplicationsApiOverview":{"type":"object","properties":{"id":{"type":"string","nullable":true},"cluster":{"type":"string","nullable":true},"status":{"type":"string","nullable":true},"tenant":{"type":"string","nullable":true},"createdBy":{"type":"string","nullable":true},"privateIp":{"type":"string","nullable":true},"publicIp":{"type":"string","nullable":true},"resourcePool":{"type":"string","nullable":true},"dns":{"type":"string","nullable":true},"sshUsername":{"type":"string","nullable":true},"applicationCatalogItemName":{"type":"string","nullable":true},"applicationCatalogItemVersionName":{"type":"string","nullable":true},"hardwarePackageName":{"type":"string","nullable":true},"persistedDirectAttachedStorage":{"type":"boolean"},"personalSharedStorage":{"type":"boolean"},"tenantSharedStorage":{"type":"boolean"}},"additionalProperties":false}}}} ``` ## POST /api/v1/servers/applications/StartApplication > Start an application that has been previously set up and provisioned, but is currently OFFLINE ```json {"openapi":"3.0.4","info":{"title":"DenvrCloud API","version":"v1"},"tags":[{"name":"servers/applications"}],"paths":{"/api/v1/servers/applications/StartApplication":{"post":{"tags":["servers/applications"],"summary":"Start an application that has been previously set up and provisioned, but is currently OFFLINE","operationId":"ApiV1ServersApplicationsStartapplicationPost","requestBody":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/ApplicationsApiCommandRequest"}},"text/json":{"schema":{"$ref":"#/components/schemas/ApplicationsApiCommandRequest"}},"application/*+json":{"schema":{"$ref":"#/components/schemas/ApplicationsApiCommandRequest"}}}},"responses":{"202":{"description":"Accepted","content":{"text/plain":{"schema":{"$ref":"#/components/schemas/ApplicationsApiCommandResponse"}},"application/json":{"schema":{"$ref":"#/components/schemas/ApplicationsApiCommandResponse"}},"text/json":{"schema":{"$ref":"#/components/schemas/ApplicationsApiCommandResponse"}}}},"400":{"description":"Bad Request"},"401":{"description":"Unauthorized"},"402":{"description":"Payment Required"},"403":{"description":"Forbidden"},"409":{"description":"Conflict"},"500":{"description":"Internal Server Error"}}}}},"components":{"schemas":{"ApplicationsApiCommandRequest":{"required":["cluster","id"],"type":"object","properties":{"id":{"minLength":1,"type":"string","description":"The application name"},"cluster":{"minLength":1,"type":"string","description":"The cluster you're operating on"}},"additionalProperties":false},"ApplicationsApiCommandResponse":{"required":["cluster","id"],"type":"object","properties":{"id":{"minLength":1,"type":"string","description":"The application name"},"cluster":{"minLength":1,"type":"string","description":"The cluster you're operating on"}},"additionalProperties":false}}}} ``` ## POST /api/v1/servers/applications/StopApplication > Stop an application that has been previously set up and provisioned, but is currently ONLINE ```json {"openapi":"3.0.4","info":{"title":"DenvrCloud API","version":"v1"},"tags":[{"name":"servers/applications"}],"paths":{"/api/v1/servers/applications/StopApplication":{"post":{"tags":["servers/applications"],"summary":"Stop an application that has been previously set up and provisioned, but is currently ONLINE","operationId":"ApiV1ServersApplicationsStopapplicationPost","requestBody":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/ApplicationsApiCommandRequest"}},"text/json":{"schema":{"$ref":"#/components/schemas/ApplicationsApiCommandRequest"}},"application/*+json":{"schema":{"$ref":"#/components/schemas/ApplicationsApiCommandRequest"}}}},"responses":{"202":{"description":"Accepted","content":{"text/plain":{"schema":{"$ref":"#/components/schemas/ApplicationsApiCommandResponse"}},"application/json":{"schema":{"$ref":"#/components/schemas/ApplicationsApiCommandResponse"}},"text/json":{"schema":{"$ref":"#/components/schemas/ApplicationsApiCommandResponse"}}}},"400":{"description":"Bad Request"},"401":{"description":"Unauthorized"},"403":{"description":"Forbidden"},"404":{"description":"Not Found"},"409":{"description":"Conflict"},"500":{"description":"Internal Server Error"}}}}},"components":{"schemas":{"ApplicationsApiCommandRequest":{"required":["cluster","id"],"type":"object","properties":{"id":{"minLength":1,"type":"string","description":"The application name"},"cluster":{"minLength":1,"type":"string","description":"The cluster you're operating on"}},"additionalProperties":false},"ApplicationsApiCommandResponse":{"required":["cluster","id"],"type":"object","properties":{"id":{"minLength":1,"type":"string","description":"The application name"},"cluster":{"minLength":1,"type":"string","description":"The cluster you're operating on"}},"additionalProperties":false}}}} ``` ## DELETE /api/v1/servers/applications/DestroyApplication > Permanently delete a specified application, effectively wiping all its data and freeing up resources for other uses ```json {"openapi":"3.0.4","info":{"title":"DenvrCloud API","version":"v1"},"tags":[{"name":"servers/applications"}],"paths":{"/api/v1/servers/applications/DestroyApplication":{"delete":{"tags":["servers/applications"],"summary":"Permanently delete a specified application, effectively wiping all its data and freeing up resources for other uses","operationId":"ApiV1ServersApplicationsDestroyapplicationDelete","parameters":[{"name":"Id","in":"query","description":"The application name","required":true,"schema":{"type":"string"}},{"name":"Cluster","in":"query","description":"The cluster you're operating on","required":true,"schema":{"type":"string"}}],"responses":{"202":{"description":"Accepted","content":{"text/plain":{"schema":{"$ref":"#/components/schemas/ApplicationsApiCommandResponse"}},"application/json":{"schema":{"$ref":"#/components/schemas/ApplicationsApiCommandResponse"}},"text/json":{"schema":{"$ref":"#/components/schemas/ApplicationsApiCommandResponse"}}}},"400":{"description":"Bad Request"},"401":{"description":"Unauthorized"},"403":{"description":"Forbidden"},"404":{"description":"Not Found"},"500":{"description":"Internal Server Error"}}}}},"components":{"schemas":{"ApplicationsApiCommandResponse":{"required":["cluster","id"],"type":"object","properties":{"id":{"minLength":1,"type":"string","description":"The application name"},"cluster":{"minLength":1,"type":"string","description":"The cluster you're operating on"}},"additionalProperties":false}}}} ``` ## GET /api/v1/servers/applications/GetApplicationRuntimeLogs > Get runtime logs for a specified application ```json {"openapi":"3.0.4","info":{"title":"DenvrCloud API","version":"v1"},"tags":[{"name":"servers/applications"}],"paths":{"/api/v1/servers/applications/GetApplicationRuntimeLogs":{"get":{"tags":["servers/applications"],"summary":"Get runtime logs for a specified application","operationId":"ApiV1ServersApplicationsGetapplicationruntimelogsGet","parameters":[{"name":"Id","in":"query","description":"The name of the application","required":true,"schema":{"type":"string"}},{"name":"Cluster","in":"query","description":"The cluster where the application is running","required":true,"schema":{"type":"string"}},{"name":"Limit","in":"query","description":"The maximum number of log entries to return.","required":true,"schema":{"maximum":10000,"minimum":1,"type":"integer","format":"int32"}}],"responses":{"200":{"description":"OK","content":{"text/plain":{"schema":{"$ref":"#/components/schemas/ApplicationsApiRuntimeLogsResponse"}},"application/json":{"schema":{"$ref":"#/components/schemas/ApplicationsApiRuntimeLogsResponse"}},"text/json":{"schema":{"$ref":"#/components/schemas/ApplicationsApiRuntimeLogsResponse"}}}},"400":{"description":"Bad Request"},"401":{"description":"Unauthorized"},"403":{"description":"Forbidden"},"404":{"description":"Not Found"},"409":{"description":"Conflict"},"500":{"description":"Internal Server Error"}}}}},"components":{"schemas":{"ApplicationsApiRuntimeLogsResponse":{"type":"object","properties":{"id":{"type":"string","description":"The name of the application","nullable":true},"cluster":{"type":"string","description":"The cluster where the application is running","nullable":true},"logs":{"type":"string","description":"The runtime logs content","nullable":true}},"additionalProperties":false}}}} ``` # Virtual machines Explore this section for comprehensive information on endpoints related to the deployment, management, and operation of virtual machines within our cloud software API environment. ## GET /api/v1/servers/virtual/GetServers > Get a list of virtual machines ```json {"openapi":"3.0.4","info":{"title":"DenvrCloud API","version":"v1"},"tags":[{"name":"servers/virtual"}],"paths":{"/api/v1/servers/virtual/GetServers":{"get":{"tags":["servers/virtual"],"summary":"Get a list of virtual machines","operationId":"ApiV1ServersVirtualGetserversGet","parameters":[{"name":"Cluster","in":"query","schema":{"type":"string"}}],"responses":{"200":{"description":"OK","content":{"text/plain":{"schema":{"$ref":"#/components/schemas/ListResultDtoOfVirtualServerDetailsItem"}},"application/json":{"schema":{"$ref":"#/components/schemas/ListResultDtoOfVirtualServerDetailsItem"}},"text/json":{"schema":{"$ref":"#/components/schemas/ListResultDtoOfVirtualServerDetailsItem"}}}},"400":{"description":"Bad Request"},"401":{"description":"Unauthorized"},"403":{"description":"Forbidden"},"404":{"description":"Not Found"},"500":{"description":"Internal Server Error"}}}}},"components":{"schemas":{"ListResultDtoOfVirtualServerDetailsItem":{"type":"object","properties":{"items":{"type":"array","items":{"$ref":"#/components/schemas/VirtualServerDetailsItem"},"nullable":true}},"additionalProperties":false},"VirtualServerDetailsItem":{"type":"object","properties":{"id":{"type":"string","description":"The name of the virtual machine","nullable":true},"namespace":{"type":"string","nullable":true},"configuration":{"type":"string","description":"A VM configuration ID","nullable":true},"storage":{"type":"integer","description":"The amount of storage attached to the VM in GB","format":"int64","nullable":true},"gpu_type":{"type":"string","description":"The specific host GPU type","nullable":true},"gpus":{"type":"integer","description":"Number of GPUs attached to the VM","format":"int32","nullable":true},"vcpus":{"type":"integer","description":"Number of vCPUs available to the VM","format":"int32","nullable":true},"memory":{"type":"integer","description":"Amount of system memory available in GB","format":"int64","nullable":true},"ip":{"type":"string","description":"The public IP address of the VM","nullable":true},"privateIp":{"type":"string","description":"The private IP address of the VM","nullable":true},"image":{"type":"string","description":"Name of the VM image used","nullable":true},"cluster":{"type":"string","description":"The cluster where the VM is allocated","nullable":true},"username":{"type":"string","description":"The user that creatd the vm","nullable":true},"tenancy_name":{"type":"string","description":"Name of the tenant where the VM has been created","nullable":true},"rpool":{"type":"string","description":"Resource pool where the VM has been created","nullable":true},"nodeSelector":{"type":"string","description":"The specific node where the VM is scheduled","nullable":true},"directAttachedStoragePersisted":{"type":"boolean"},"status":{"type":"string","description":"The status of the VM (e.g. 'PLANNED', 'PENDING' 'PENDING_RESOURCES', 'PENDING_READINESS', 'ONLINE', 'OFFLINE')","nullable":true},"storageType":{"type":"string","nullable":true},"rootDiskSize":{"type":"string","nullable":true},"lastUpdated":{"type":"string","format":"date-time"}},"additionalProperties":false}}}} ``` ## GET /api/v1/servers/virtual/GetServer > Get detailed information about a specific virtual machine ```json {"openapi":"3.0.4","info":{"title":"DenvrCloud API","version":"v1"},"tags":[{"name":"servers/virtual"}],"paths":{"/api/v1/servers/virtual/GetServer":{"get":{"tags":["servers/virtual"],"summary":"Get detailed information about a specific virtual machine","operationId":"ApiV1ServersVirtualGetserverGet","parameters":[{"name":"Id","in":"query","description":"The virtual machine id","required":true,"schema":{"type":"string"}},{"name":"Namespace","in":"query","description":"The namespace/vpc where the virtual machine lives. Default one is same as tenant name.","required":true,"schema":{"type":"string"}},{"name":"Cluster","in":"query","description":"The cluster you're operating on","required":true,"schema":{"type":"string"}}],"responses":{"200":{"description":"OK","content":{"text/plain":{"schema":{"$ref":"#/components/schemas/VirtualServerDetailsItem"}},"application/json":{"schema":{"$ref":"#/components/schemas/VirtualServerDetailsItem"}},"text/json":{"schema":{"$ref":"#/components/schemas/VirtualServerDetailsItem"}}}},"400":{"description":"Bad Request"},"401":{"description":"Unauthorized"},"403":{"description":"Forbidden"},"404":{"description":"Not Found"},"500":{"description":"Internal Server Error"}}}}},"components":{"schemas":{"VirtualServerDetailsItem":{"type":"object","properties":{"id":{"type":"string","description":"The name of the virtual machine","nullable":true},"namespace":{"type":"string","nullable":true},"configuration":{"type":"string","description":"A VM configuration ID","nullable":true},"storage":{"type":"integer","description":"The amount of storage attached to the VM in GB","format":"int64","nullable":true},"gpu_type":{"type":"string","description":"The specific host GPU type","nullable":true},"gpus":{"type":"integer","description":"Number of GPUs attached to the VM","format":"int32","nullable":true},"vcpus":{"type":"integer","description":"Number of vCPUs available to the VM","format":"int32","nullable":true},"memory":{"type":"integer","description":"Amount of system memory available in GB","format":"int64","nullable":true},"ip":{"type":"string","description":"The public IP address of the VM","nullable":true},"privateIp":{"type":"string","description":"The private IP address of the VM","nullable":true},"image":{"type":"string","description":"Name of the VM image used","nullable":true},"cluster":{"type":"string","description":"The cluster where the VM is allocated","nullable":true},"username":{"type":"string","description":"The user that creatd the vm","nullable":true},"tenancy_name":{"type":"string","description":"Name of the tenant where the VM has been created","nullable":true},"rpool":{"type":"string","description":"Resource pool where the VM has been created","nullable":true},"nodeSelector":{"type":"string","description":"The specific node where the VM is scheduled","nullable":true},"directAttachedStoragePersisted":{"type":"boolean"},"status":{"type":"string","description":"The status of the VM (e.g. 'PLANNED', 'PENDING' 'PENDING_RESOURCES', 'PENDING_READINESS', 'ONLINE', 'OFFLINE')","nullable":true},"storageType":{"type":"string","nullable":true},"rootDiskSize":{"type":"string","nullable":true},"lastUpdated":{"type":"string","format":"date-time"}},"additionalProperties":false}}}} ``` ## POST /api/v1/servers/virtual/CreateServer > Create a new virtual machine using a pre-defined configuration ```json {"openapi":"3.0.4","info":{"title":"DenvrCloud API","version":"v1"},"tags":[{"name":"servers/virtual"}],"paths":{"/api/v1/servers/virtual/CreateServer":{"post":{"tags":["servers/virtual"],"summary":"Create a new virtual machine using a pre-defined configuration","operationId":"ApiV1ServersVirtualCreateserverPost","requestBody":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/CreateVirtualServerInput"}},"text/json":{"schema":{"$ref":"#/components/schemas/CreateVirtualServerInput"}},"application/*+json":{"schema":{"$ref":"#/components/schemas/CreateVirtualServerInput"}}}},"responses":{"202":{"description":"Accepted","content":{"text/plain":{"schema":{"$ref":"#/components/schemas/VirtualServerDetailsItem"}},"application/json":{"schema":{"$ref":"#/components/schemas/VirtualServerDetailsItem"}},"text/json":{"schema":{"$ref":"#/components/schemas/VirtualServerDetailsItem"}}}},"400":{"description":"Bad Request"},"401":{"description":"Unauthorized"},"403":{"description":"Forbidden"},"404":{"description":"Not Found"},"500":{"description":"Internal Server Error"},"503":{"description":"Service Unavailable"}}}}},"components":{"schemas":{"CreateVirtualServerInput":{"required":["cluster","configuration","ssh_keys","vpc"],"type":"object","properties":{"name":{"maxLength":52,"minLength":0,"type":"string","description":"Name of virtual server to be created. If not provided, name will be auto-generated.\nMust be 52 characters or less.","nullable":true},"rpool":{"type":"string","description":"Name of the pool to be used. If not provided, first pool assigned to a tenant will be used. In case of no pool assigned, 'on-demand' will be used.","nullable":true},"vpc":{"minLength":1,"type":"string","description":"Name of the VPC to be used. Usually this will match the tenant name."},"configuration":{"minLength":1,"type":"string","description":"Name of the configuration to be used. For possible values, refer to the otput of api/v1/servers/virtual/GetConfigurations, field 'name' DenvrDashboard.Servers.Dtos.ServerConfiguration.Name"},"cluster":{"minLength":1,"type":"string","description":"Cluster to be used. For possible values, refer to the otput of api/v1/clusters/GetAll\"/>"},"ssh_keys":{"minItems":1,"type":"array","items":{"type":"string"}},"snapshotName":{"type":"string","description":"Snapshot name.","nullable":true},"tenantCustomImageFileName":{"type":"string","description":"Tenant custom image file name. When set, the VM is deployed from a tenant custom image.","nullable":true},"operatingSystemImage":{"type":"string","description":"Name of the Operating System image to be used.","nullable":true},"personalStorageMountPath":{"type":"string","description":"Personal storage file system mount path.","nullable":true},"tenantSharedAdditionalStorage":{"type":"string","description":"Tenant shared storage file system mount path.","nullable":true},"persistStorage":{"type":"boolean","description":"Whether direct attached storage should be persistant or ephemeral."},"directStorageMountPath":{"type":"string","description":"Direct attached storage mount path.","nullable":true},"blockStorageNames":{"type":"array","items":{"type":"string"},"description":"Names of additional block storage volumes to attach at deploy time.","nullable":true},"rootDiskSize":{"type":"integer","description":"Size of root disk to be created (Gi).","format":"int32"},"selectedNode":{"type":"string","description":"Specific node name to target for VM deployment. \nUsed for non-on-demand resource pools to allow node-specific scheduling.\nMaps to Kubernetes nodeSelector with kubernetes.io/hostname label.","nullable":true}},"additionalProperties":false},"VirtualServerDetailsItem":{"type":"object","properties":{"id":{"type":"string","description":"The name of the virtual machine","nullable":true},"namespace":{"type":"string","nullable":true},"configuration":{"type":"string","description":"A VM configuration ID","nullable":true},"storage":{"type":"integer","description":"The amount of storage attached to the VM in GB","format":"int64","nullable":true},"gpu_type":{"type":"string","description":"The specific host GPU type","nullable":true},"gpus":{"type":"integer","description":"Number of GPUs attached to the VM","format":"int32","nullable":true},"vcpus":{"type":"integer","description":"Number of vCPUs available to the VM","format":"int32","nullable":true},"memory":{"type":"integer","description":"Amount of system memory available in GB","format":"int64","nullable":true},"ip":{"type":"string","description":"The public IP address of the VM","nullable":true},"privateIp":{"type":"string","description":"The private IP address of the VM","nullable":true},"image":{"type":"string","description":"Name of the VM image used","nullable":true},"cluster":{"type":"string","description":"The cluster where the VM is allocated","nullable":true},"username":{"type":"string","description":"The user that creatd the vm","nullable":true},"tenancy_name":{"type":"string","description":"Name of the tenant where the VM has been created","nullable":true},"rpool":{"type":"string","description":"Resource pool where the VM has been created","nullable":true},"nodeSelector":{"type":"string","description":"The specific node where the VM is scheduled","nullable":true},"directAttachedStoragePersisted":{"type":"boolean"},"status":{"type":"string","description":"The status of the VM (e.g. 'PLANNED', 'PENDING' 'PENDING_RESOURCES', 'PENDING_READINESS', 'ONLINE', 'OFFLINE')","nullable":true},"storageType":{"type":"string","nullable":true},"rootDiskSize":{"type":"string","nullable":true},"lastUpdated":{"type":"string","format":"date-time"}},"additionalProperties":false}}}} ``` ## POST /api/v1/servers/virtual/StartServer > Start a virtual machine that has been previously set up and provisioned, but is currently OFFLINE ```json {"openapi":"3.0.4","info":{"title":"DenvrCloud API","version":"v1"},"tags":[{"name":"servers/virtual"}],"paths":{"/api/v1/servers/virtual/StartServer":{"post":{"tags":["servers/virtual"],"summary":"Start a virtual machine that has been previously set up and provisioned, but is currently OFFLINE","operationId":"ApiV1ServersVirtualStartserverPost","requestBody":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/ServerCommandInput"}},"text/json":{"schema":{"$ref":"#/components/schemas/ServerCommandInput"}},"application/*+json":{"schema":{"$ref":"#/components/schemas/ServerCommandInput"}}}},"responses":{"202":{"description":"Accepted","content":{"text/plain":{"schema":{"$ref":"#/components/schemas/ServerCommandOutput"}},"application/json":{"schema":{"$ref":"#/components/schemas/ServerCommandOutput"}},"text/json":{"schema":{"$ref":"#/components/schemas/ServerCommandOutput"}}}},"400":{"description":"Bad Request"},"401":{"description":"Unauthorized"},"403":{"description":"Forbidden"},"404":{"description":"Not Found"},"500":{"description":"Internal Server Error"},"503":{"description":"Service Unavailable"}}}}},"components":{"schemas":{"ServerCommandInput":{"required":["cluster","id","namespace"],"type":"object","properties":{"id":{"minLength":1,"type":"string","description":"The virtual machine id"},"namespace":{"minLength":1,"type":"string","description":"The namespace/vpc where the virtual machine lives. Default one is same as tenant name."},"cluster":{"minLength":1,"type":"string","description":"The cluster you're operating on"}},"additionalProperties":false},"ServerCommandOutput":{"type":"object","properties":{"id":{"type":"string","nullable":true},"cluster":{"type":"string","nullable":true},"status":{"type":"string","nullable":true}},"additionalProperties":false}}}} ``` ## POST /api/v1/servers/virtual/StopServer > Stop a virtual machine, ensuring a secure and orderly shutdown of its operations within the cloud environment ```json {"openapi":"3.0.4","info":{"title":"DenvrCloud API","version":"v1"},"tags":[{"name":"servers/virtual"}],"paths":{"/api/v1/servers/virtual/StopServer":{"post":{"tags":["servers/virtual"],"summary":"Stop a virtual machine, ensuring a secure and orderly shutdown of its operations within the cloud environment","operationId":"ApiV1ServersVirtualStopserverPost","requestBody":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/ServerCommandInput"}},"text/json":{"schema":{"$ref":"#/components/schemas/ServerCommandInput"}},"application/*+json":{"schema":{"$ref":"#/components/schemas/ServerCommandInput"}}}},"responses":{"202":{"description":"Accepted","content":{"text/plain":{"schema":{"$ref":"#/components/schemas/ServerCommandOutput"}},"application/json":{"schema":{"$ref":"#/components/schemas/ServerCommandOutput"}},"text/json":{"schema":{"$ref":"#/components/schemas/ServerCommandOutput"}}}},"400":{"description":"Bad Request"},"401":{"description":"Unauthorized"},"403":{"description":"Forbidden"},"404":{"description":"Not Found"},"500":{"description":"Internal Server Error"}}}}},"components":{"schemas":{"ServerCommandInput":{"required":["cluster","id","namespace"],"type":"object","properties":{"id":{"minLength":1,"type":"string","description":"The virtual machine id"},"namespace":{"minLength":1,"type":"string","description":"The namespace/vpc where the virtual machine lives. Default one is same as tenant name."},"cluster":{"minLength":1,"type":"string","description":"The cluster you're operating on"}},"additionalProperties":false},"ServerCommandOutput":{"type":"object","properties":{"id":{"type":"string","nullable":true},"cluster":{"type":"string","nullable":true},"status":{"type":"string","nullable":true}},"additionalProperties":false}}}} ``` ## DELETE /api/v1/servers/virtual/DestroyServer > Permanently delete a specified virtual machine, effectively wiping all its data and freeing up resources for other uses ```json {"openapi":"3.0.4","info":{"title":"DenvrCloud API","version":"v1"},"tags":[{"name":"servers/virtual"}],"paths":{"/api/v1/servers/virtual/DestroyServer":{"delete":{"tags":["servers/virtual"],"summary":"Permanently delete a specified virtual machine, effectively wiping all its data and freeing up resources for other uses","operationId":"ApiV1ServersVirtualDestroyserverDelete","parameters":[{"name":"DeleteSnapshots","in":"query","description":"Should also delete snapshots with virtual machine.","schema":{"type":"boolean"}},{"name":"Id","in":"query","description":"The virtual machine id","required":true,"schema":{"type":"string"}},{"name":"Namespace","in":"query","description":"The namespace/vpc where the virtual machine lives. Default one is same as tenant name.","required":true,"schema":{"type":"string"}},{"name":"Cluster","in":"query","description":"The cluster you're operating on","required":true,"schema":{"type":"string"}}],"responses":{"202":{"description":"Accepted","content":{"text/plain":{"schema":{"$ref":"#/components/schemas/ServerCommandOutput"}},"application/json":{"schema":{"$ref":"#/components/schemas/ServerCommandOutput"}},"text/json":{"schema":{"$ref":"#/components/schemas/ServerCommandOutput"}}}},"400":{"description":"Bad Request"},"401":{"description":"Unauthorized"},"403":{"description":"Forbidden"},"404":{"description":"Not Found"},"409":{"description":"Conflict"},"500":{"description":"Internal Server Error"}}}}},"components":{"schemas":{"ServerCommandOutput":{"type":"object","properties":{"id":{"type":"string","nullable":true},"cluster":{"type":"string","nullable":true},"status":{"type":"string","nullable":true}},"additionalProperties":false}}}} ``` ## GET /api/v1/servers/virtual/GetConfigurations > Get detailed information on available configurations for virtual machines ```json {"openapi":"3.0.4","info":{"title":"DenvrCloud API","version":"v1"},"tags":[{"name":"servers/virtual"}],"paths":{"/api/v1/servers/virtual/GetConfigurations":{"get":{"tags":["servers/virtual"],"summary":"Get detailed information on available configurations for virtual machines","operationId":"ApiV1ServersVirtualGetconfigurationsGet","responses":{"200":{"description":"OK","content":{"text/plain":{"schema":{"$ref":"#/components/schemas/ListResultDtoOfServerConfiguration"}},"application/json":{"schema":{"$ref":"#/components/schemas/ListResultDtoOfServerConfiguration"}},"text/json":{"schema":{"$ref":"#/components/schemas/ListResultDtoOfServerConfiguration"}}}},"400":{"description":"Bad Request"},"401":{"description":"Unauthorized"},"403":{"description":"Forbidden"},"500":{"description":"Internal Server Error"}}}}},"components":{"schemas":{"ListResultDtoOfServerConfiguration":{"type":"object","properties":{"items":{"type":"array","items":{"$ref":"#/components/schemas/ServerConfiguration"},"nullable":true}},"additionalProperties":false},"ServerConfiguration":{"type":"object","properties":{"id":{"type":"integer","format":"int32"},"user_friendly_name":{"type":"string","nullable":true},"name":{"type":"string","nullable":true},"description":{"type":"string","nullable":true},"os_version":{"type":"string","nullable":true,"deprecated":true},"os_type":{"type":"string","nullable":true,"deprecated":true},"storage":{"type":"integer","format":"int64"},"gpu_type":{"type":"string","nullable":true,"deprecated":true},"gpu_family":{"type":"string","nullable":true,"deprecated":true},"gpu_brand":{"type":"string","nullable":true,"deprecated":true},"gpu_name":{"type":"string","nullable":true,"deprecated":true},"type":{"type":"string","nullable":true},"brand_family":{"type":"string","nullable":true},"brand":{"type":"string","nullable":true},"text_name":{"type":"string","nullable":true},"gpus":{"type":"integer","format":"int32"},"vcpus":{"type":"integer","format":"int32"},"memory":{"type":"integer","format":"int64"},"price":{"type":"number","format":"double"},"compute_network":{"type":"string","nullable":true},"is_gpu_platform":{"type":"boolean"},"clusters":{"type":"array","items":{"type":"string"},"nullable":true}},"additionalProperties":false}}}} ``` ## GET /api/v1/servers/virtual/GetAvailability > Get information about the current availability of different virtual machine configurations ```json {"openapi":"3.0.4","info":{"title":"DenvrCloud API","version":"v1"},"tags":[{"name":"servers/virtual"}],"paths":{"/api/v1/servers/virtual/GetAvailability":{"get":{"tags":["servers/virtual"],"summary":"Get information about the current availability of different virtual machine configurations","operationId":"ApiV1ServersVirtualGetavailabilityGet","parameters":[{"name":"cluster","in":"query","description":"","required":true,"schema":{"type":"string"}},{"name":"resourcePool","in":"query","description":"","schema":{"type":"string"}},{"name":"reportNodes","in":"query","description":"controls if Count and MaxCount is calculated and returned in the response. If they are not needed, use 'false' to improve response time of the endpoint.","schema":{"type":"boolean","default":true}}],"responses":{"200":{"description":"OK","content":{"text/plain":{"schema":{"$ref":"#/components/schemas/ListResultDtoOfServerAvailability"}},"application/json":{"schema":{"$ref":"#/components/schemas/ListResultDtoOfServerAvailability"}},"text/json":{"schema":{"$ref":"#/components/schemas/ListResultDtoOfServerAvailability"}}}},"400":{"description":"Bad Request"},"401":{"description":"Unauthorized"},"403":{"description":"Forbidden"},"500":{"description":"Internal Server Error"}}}}},"components":{"schemas":{"ListResultDtoOfServerAvailability":{"type":"object","properties":{"items":{"type":"array","items":{"$ref":"#/components/schemas/ServerAvailability"},"nullable":true}},"additionalProperties":false},"ServerAvailability":{"type":"object","properties":{"configuration":{"type":"string","nullable":true},"cluster":{"type":"string","nullable":true},"rpool":{"type":"string","nullable":true},"type":{"type":"string","nullable":true,"deprecated":true},"price":{"type":"number","format":"double"},"available":{"type":"boolean"},"count":{"type":"integer","description":"number of servers that can be created with this configuration.","format":"int32"},"maxCount":{"type":"integer","description":"maximum number of servers that can be created with this configuration. 0 if resource pool is on-demand.","format":"int32"},"availableNodeNames":{"type":"array","items":{"type":"string"},"description":"List of node names where this configuration is available. Omitted for on-demand resource pools","nullable":true}},"additionalProperties":false}}}} ``` ## GET /api/v1/servers/virtual/GetVirtualMachineBootLogs > Get boot logs for a specific virtual machine ```json {"openapi":"3.0.4","info":{"title":"DenvrCloud API","version":"v1"},"tags":[{"name":"servers/virtual"}],"paths":{"/api/v1/servers/virtual/GetVirtualMachineBootLogs":{"get":{"tags":["servers/virtual"],"summary":"Get boot logs for a specific virtual machine","operationId":"ApiV1ServersVirtualGetvirtualmachinebootlogsGet","parameters":[{"name":"Id","in":"query","description":"The virtual machine id","required":true,"schema":{"type":"string"}},{"name":"Namespace","in":"query","description":"The namespace/vpc where the virtual machine lives. Default one is same as tenant name.","required":true,"schema":{"type":"string"}},{"name":"Cluster","in":"query","description":"The cluster you're operating on","required":true,"schema":{"type":"string"}},{"name":"Limit","in":"query","description":"The maximum number of log entries to return. Defaults to 2000.","required":true,"schema":{"maximum":10000,"minimum":1,"type":"integer","format":"int32"}}],"responses":{"200":{"description":"OK","content":{"text/plain":{"schema":{"$ref":"#/components/schemas/ServerBootLogsOutput"}},"application/json":{"schema":{"$ref":"#/components/schemas/ServerBootLogsOutput"}},"text/json":{"schema":{"$ref":"#/components/schemas/ServerBootLogsOutput"}}}},"400":{"description":"Bad Request"},"401":{"description":"Unauthorized"},"403":{"description":"Forbidden"},"404":{"description":"Not Found"},"409":{"description":"Conflict"},"500":{"description":"Internal Server Error"}}}}},"components":{"schemas":{"ServerBootLogsOutput":{"type":"object","properties":{"id":{"type":"string","nullable":true},"namespace":{"type":"string","nullable":true},"cluster":{"type":"string","nullable":true},"bootLogs":{"type":"string","nullable":true}},"additionalProperties":false}}}} ``` # VPCs Explore this section for comprehensive information on endpoints related to the deployment, management, and operation of Virtual Pri within our cloud software API environment. ## GET /api/v1/vpcs/GetVpcs > Get a list of VPCs ```json {"openapi":"3.0.4","info":{"title":"DenvrCloud API","version":"v1"},"tags":[{"name":"vpcs"}],"paths":{"/api/v1/vpcs/GetVpcs":{"get":{"tags":["vpcs"],"summary":"Get a list of VPCs","operationId":"ApiV1VpcsGetvpcsGet","parameters":[{"name":"cluster","in":"query","schema":{"type":"string"}}],"responses":{"200":{"description":"OK","content":{"text/plain":{"schema":{"$ref":"#/components/schemas/ListResultDtoOfVpcDetailsItem"}},"application/json":{"schema":{"$ref":"#/components/schemas/ListResultDtoOfVpcDetailsItem"}},"text/json":{"schema":{"$ref":"#/components/schemas/ListResultDtoOfVpcDetailsItem"}}}},"400":{"description":"Bad Request"},"401":{"description":"Unauthorized"},"403":{"description":"Forbidden"},"500":{"description":"Internal Server Error"}}}}},"components":{"schemas":{"ListResultDtoOfVpcDetailsItem":{"type":"object","properties":{"items":{"type":"array","items":{"$ref":"#/components/schemas/VpcDetailsItem"},"nullable":true}},"additionalProperties":false},"VpcDetailsItem":{"type":"object","properties":{"id":{"type":"string","nullable":true},"name":{"type":"string","nullable":true},"cluster":{"type":"string","nullable":true},"tenancyName":{"type":"string","nullable":true},"ipRange":{"type":"string","nullable":true},"createdAt":{"type":"string","format":"date-time"},"blockIntraVpcComms":{"type":"boolean"},"isDefault":{"type":"boolean"}},"additionalProperties":false}}}} ``` ## GET /api/v1/vpcs/GetVpc > Get detailed information about a specific VPC ```json {"openapi":"3.0.4","info":{"title":"DenvrCloud API","version":"v1"},"tags":[{"name":"vpcs"}],"paths":{"/api/v1/vpcs/GetVpc":{"get":{"tags":["vpcs"],"summary":"Get detailed information about a specific VPC","operationId":"ApiV1VpcsGetvpcGet","parameters":[{"name":"Id","in":"query","description":"Unique identifier for a resource within the cluster","required":true,"schema":{"type":"string"}},{"name":"Cluster","in":"query","description":"The cluster you're operating on","required":true,"schema":{"type":"string"}}],"responses":{"200":{"description":"OK","content":{"text/plain":{"schema":{"$ref":"#/components/schemas/VpcDetailsItem"}},"application/json":{"schema":{"$ref":"#/components/schemas/VpcDetailsItem"}},"text/json":{"schema":{"$ref":"#/components/schemas/VpcDetailsItem"}}}},"400":{"description":"Bad Request"},"401":{"description":"Unauthorized"},"403":{"description":"Forbidden"},"500":{"description":"Internal Server Error"}}}}},"components":{"schemas":{"VpcDetailsItem":{"type":"object","properties":{"id":{"type":"string","nullable":true},"name":{"type":"string","nullable":true},"cluster":{"type":"string","nullable":true},"tenancyName":{"type":"string","nullable":true},"ipRange":{"type":"string","nullable":true},"createdAt":{"type":"string","format":"date-time"},"blockIntraVpcComms":{"type":"boolean"},"isDefault":{"type":"boolean"}},"additionalProperties":false}}}} ``` ## POST /api/v1/vpcs/CreateVpc > Create a new VPC ```json {"openapi":"3.0.4","info":{"title":"DenvrCloud API","version":"v1"},"tags":[{"name":"vpcs"}],"paths":{"/api/v1/vpcs/CreateVpc":{"post":{"tags":["vpcs"],"summary":"Create a new VPC","operationId":"ApiV1VpcsCreatevpcPost","requestBody":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/CreateVpcInputExt"}},"text/json":{"schema":{"$ref":"#/components/schemas/CreateVpcInputExt"}},"application/*+json":{"schema":{"$ref":"#/components/schemas/CreateVpcInputExt"}}}},"responses":{"200":{"description":"OK","content":{"text/plain":{"schema":{"$ref":"#/components/schemas/VpcDetailsItem"}},"application/json":{"schema":{"$ref":"#/components/schemas/VpcDetailsItem"}},"text/json":{"schema":{"$ref":"#/components/schemas/VpcDetailsItem"}}}},"400":{"description":"Bad Request"},"401":{"description":"Unauthorized"},"403":{"description":"Forbidden"},"500":{"description":"Internal Server Error"}}}}},"components":{"schemas":{"CreateVpcInputExt":{"required":["cluster","name"],"type":"object","properties":{"name":{"minLength":1,"type":"string","description":"name of the VPC. should start with {tenancyName}- \nin case of creating default VPC, name should be {tenancyName}"},"blockIntraVpcComms":{"type":"boolean","description":"if set to true, this VPC will block any intra-VPC communications\nif ommited, default is false"},"isDefault":{"type":"boolean","description":"if set to true, this VPC will be the default VPC for the cluster\nonly one VPC can be default per cluster. \nif omitted, default is false"},"cluster":{"minLength":1,"type":"string","description":"The cluster you're operating on"}},"additionalProperties":false},"VpcDetailsItem":{"type":"object","properties":{"id":{"type":"string","nullable":true},"name":{"type":"string","nullable":true},"cluster":{"type":"string","nullable":true},"tenancyName":{"type":"string","nullable":true},"ipRange":{"type":"string","nullable":true},"createdAt":{"type":"string","format":"date-time"},"blockIntraVpcComms":{"type":"boolean"},"isDefault":{"type":"boolean"}},"additionalProperties":false}}}} ``` ## DELETE /api/v1/vpcs/DestroyVpc > Destroy a VPC ```json {"openapi":"3.0.4","info":{"title":"DenvrCloud API","version":"v1"},"tags":[{"name":"vpcs"}],"paths":{"/api/v1/vpcs/DestroyVpc":{"delete":{"tags":["vpcs"],"summary":"Destroy a VPC","operationId":"ApiV1VpcsDestroyvpcDelete","parameters":[{"name":"Id","in":"query","description":"Unique identifier for a resource within the cluster","required":true,"schema":{"type":"string"}},{"name":"Cluster","in":"query","description":"The cluster you're operating on","required":true,"schema":{"type":"string"}}],"responses":{"200":{"description":"OK","content":{"text/plain":{"schema":{"$ref":"#/components/schemas/VpcDetailsItem"}},"application/json":{"schema":{"$ref":"#/components/schemas/VpcDetailsItem"}},"text/json":{"schema":{"$ref":"#/components/schemas/VpcDetailsItem"}}}},"400":{"description":"Bad Request"},"401":{"description":"Unauthorized"},"403":{"description":"Forbidden"},"500":{"description":"Internal Server Error"}}}}},"components":{"schemas":{"VpcDetailsItem":{"type":"object","properties":{"id":{"type":"string","nullable":true},"name":{"type":"string","nullable":true},"cluster":{"type":"string","nullable":true},"tenancyName":{"type":"string","nullable":true},"ipRange":{"type":"string","nullable":true},"createdAt":{"type":"string","format":"date-time"},"blockIntraVpcComms":{"type":"boolean"},"isDefault":{"type":"boolean"}},"additionalProperties":false}}}} ``` # Bare metal Explore this section for detailed information on endpoints related to the management and operation of bare metal servers within our cloud software API environment. ## GET /api/v1/servers/metal/GetHost > Get detailed information about a specific metal host ```json {"openapi":"3.0.4","info":{"title":"DenvrCloud API","version":"v1"},"tags":[{"name":"servers/metal"}],"paths":{"/api/v1/servers/metal/GetHost":{"get":{"tags":["servers/metal"],"summary":"Get detailed information about a specific metal host","operationId":"ApiV1ServersMetalGethostGet","parameters":[{"name":"Id","in":"query","description":"Unique identifier for a resource within the cluster","required":true,"schema":{"type":"string"}},{"name":"Cluster","in":"query","description":"The cluster you're operating on","required":true,"schema":{"type":"string"}}],"responses":{"200":{"description":"OK","content":{"text/plain":{"schema":{"$ref":"#/components/schemas/BareMetalHostDetailsItem"}},"application/json":{"schema":{"$ref":"#/components/schemas/BareMetalHostDetailsItem"}},"text/json":{"schema":{"$ref":"#/components/schemas/BareMetalHostDetailsItem"}}}},"400":{"description":"Bad Request"},"401":{"description":"Unauthorized"},"403":{"description":"Forbidden"},"500":{"description":"Internal Server Error"}}}}},"components":{"schemas":{"BareMetalHostDetailsItem":{"type":"object","properties":{"id":{"type":"string","description":"The bare metal id, unique identifier","nullable":true},"cluster":{"type":"string","description":"The cluster where the bare metal host is allocated","nullable":true},"tenancyName":{"type":"string","description":"Name of the tenant where the node has been allocated","nullable":true},"nodeType":{"type":"string","description":"The specific host node type","nullable":true},"image":{"type":"string","description":"The image used to provision the host","nullable":true},"privateIp":{"type":"string","description":"private IP address of the host","nullable":true},"publicIp":{"type":"string","description":"public IP address of the host","nullable":true},"provisionedHostname":{"type":"string","description":"host name provisioned by the system","nullable":true},"operationalStatus":{"type":"string","description":"operational status of the host","nullable":true},"poweredOn":{"type":"boolean","description":"true if the host is powered on"},"provisioningState":{"type":"string","description":"provisioning status of the host","nullable":true}},"additionalProperties":false}}}} ``` ## GET /api/v1/servers/metal/GetHosts > Get a list of bare metal hosts in a cluster ```json {"openapi":"3.0.4","info":{"title":"DenvrCloud API","version":"v1"},"tags":[{"name":"servers/metal"}],"paths":{"/api/v1/servers/metal/GetHosts":{"get":{"tags":["servers/metal"],"summary":"Get a list of bare metal hosts in a cluster","operationId":"ApiV1ServersMetalGethostsGet","parameters":[{"name":"Cluster","in":"query","description":"","schema":{"type":"string"}}],"responses":{"200":{"description":"OK","content":{"text/plain":{"schema":{"$ref":"#/components/schemas/ListResultDtoOfBareMetalHostDetailsItem"}},"application/json":{"schema":{"$ref":"#/components/schemas/ListResultDtoOfBareMetalHostDetailsItem"}},"text/json":{"schema":{"$ref":"#/components/schemas/ListResultDtoOfBareMetalHostDetailsItem"}}}},"400":{"description":"Bad Request"},"401":{"description":"Unauthorized"},"403":{"description":"Forbidden"},"500":{"description":"Internal Server Error"}}}}},"components":{"schemas":{"ListResultDtoOfBareMetalHostDetailsItem":{"type":"object","properties":{"items":{"type":"array","items":{"$ref":"#/components/schemas/BareMetalHostDetailsItem"},"nullable":true}},"additionalProperties":false},"BareMetalHostDetailsItem":{"type":"object","properties":{"id":{"type":"string","description":"The bare metal id, unique identifier","nullable":true},"cluster":{"type":"string","description":"The cluster where the bare metal host is allocated","nullable":true},"tenancyName":{"type":"string","description":"Name of the tenant where the node has been allocated","nullable":true},"nodeType":{"type":"string","description":"The specific host node type","nullable":true},"image":{"type":"string","description":"The image used to provision the host","nullable":true},"privateIp":{"type":"string","description":"private IP address of the host","nullable":true},"publicIp":{"type":"string","description":"public IP address of the host","nullable":true},"provisionedHostname":{"type":"string","description":"host name provisioned by the system","nullable":true},"operationalStatus":{"type":"string","description":"operational status of the host","nullable":true},"poweredOn":{"type":"boolean","description":"true if the host is powered on"},"provisioningState":{"type":"string","description":"provisioning status of the host","nullable":true}},"additionalProperties":false}}}} ``` ## POST /api/v1/servers/metal/RebootHost > Reboot the bare metal host ```json {"openapi":"3.0.4","info":{"title":"DenvrCloud API","version":"v1"},"tags":[{"name":"servers/metal"}],"paths":{"/api/v1/servers/metal/RebootHost":{"post":{"tags":["servers/metal"],"summary":"Reboot the bare metal host","operationId":"ApiV1ServersMetalReboothostPost","requestBody":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/BareMetalCommandInput"}},"text/json":{"schema":{"$ref":"#/components/schemas/BareMetalCommandInput"}},"application/*+json":{"schema":{"$ref":"#/components/schemas/BareMetalCommandInput"}}}},"responses":{"200":{"description":"OK","content":{"text/plain":{"schema":{"$ref":"#/components/schemas/BareMetalHostDetailsItem"}},"application/json":{"schema":{"$ref":"#/components/schemas/BareMetalHostDetailsItem"}},"text/json":{"schema":{"$ref":"#/components/schemas/BareMetalHostDetailsItem"}}}},"400":{"description":"Bad Request"},"401":{"description":"Unauthorized"},"403":{"description":"Forbidden"},"422":{"description":"Unprocessable Content"},"500":{"description":"Internal Server Error"}}}}},"components":{"schemas":{"BareMetalCommandInput":{"required":["cluster","id"],"type":"object","properties":{"id":{"minLength":1,"type":"string","description":"Unique identifier for a resource within the cluster"},"cluster":{"minLength":1,"type":"string","description":"The cluster you're operating on"}},"additionalProperties":false},"BareMetalHostDetailsItem":{"type":"object","properties":{"id":{"type":"string","description":"The bare metal id, unique identifier","nullable":true},"cluster":{"type":"string","description":"The cluster where the bare metal host is allocated","nullable":true},"tenancyName":{"type":"string","description":"Name of the tenant where the node has been allocated","nullable":true},"nodeType":{"type":"string","description":"The specific host node type","nullable":true},"image":{"type":"string","description":"The image used to provision the host","nullable":true},"privateIp":{"type":"string","description":"private IP address of the host","nullable":true},"publicIp":{"type":"string","description":"public IP address of the host","nullable":true},"provisionedHostname":{"type":"string","description":"host name provisioned by the system","nullable":true},"operationalStatus":{"type":"string","description":"operational status of the host","nullable":true},"poweredOn":{"type":"boolean","description":"true if the host is powered on"},"provisioningState":{"type":"string","description":"provisioning status of the host","nullable":true}},"additionalProperties":false}}}} ``` ## POST /api/v1/servers/metal/ReprovisionHost > Reprovision the bare metal host ```json {"openapi":"3.0.4","info":{"title":"DenvrCloud API","version":"v1"},"tags":[{"name":"servers/metal"}],"paths":{"/api/v1/servers/metal/ReprovisionHost":{"post":{"tags":["servers/metal"],"summary":"Reprovision the bare metal host","operationId":"ApiV1ServersMetalReprovisionhostPost","requestBody":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/BareMetalReprovisionHostInput"}},"text/json":{"schema":{"$ref":"#/components/schemas/BareMetalReprovisionHostInput"}},"application/*+json":{"schema":{"$ref":"#/components/schemas/BareMetalReprovisionHostInput"}}}},"responses":{"200":{"description":"OK","content":{"text/plain":{"schema":{"$ref":"#/components/schemas/BareMetalHostDetailsItem"}},"application/json":{"schema":{"$ref":"#/components/schemas/BareMetalHostDetailsItem"}},"text/json":{"schema":{"$ref":"#/components/schemas/BareMetalHostDetailsItem"}}}},"400":{"description":"Bad Request"},"401":{"description":"Unauthorized"},"403":{"description":"Forbidden"},"422":{"description":"Unprocessable Content"},"500":{"description":"Internal Server Error"}}}}},"components":{"schemas":{"BareMetalReprovisionHostInput":{"required":["cluster","id"],"type":"object","properties":{"id":{"minLength":1,"type":"string","description":"Unique identifier for a resource within the cluster"},"cluster":{"minLength":1,"type":"string","description":"The cluster you're operating on"},"imageUrl":{"type":"string","description":"The URL to the image to use for the host","nullable":true},"imageChecksum":{"type":"string","description":"The checksum url of the image to use for the host","nullable":true},"cloudInitBase64":{"type":"string","description":"Base64 encoded cloud-init data yaml file to use for the host","nullable":true}},"additionalProperties":false},"BareMetalHostDetailsItem":{"type":"object","properties":{"id":{"type":"string","description":"The bare metal id, unique identifier","nullable":true},"cluster":{"type":"string","description":"The cluster where the bare metal host is allocated","nullable":true},"tenancyName":{"type":"string","description":"Name of the tenant where the node has been allocated","nullable":true},"nodeType":{"type":"string","description":"The specific host node type","nullable":true},"image":{"type":"string","description":"The image used to provision the host","nullable":true},"privateIp":{"type":"string","description":"private IP address of the host","nullable":true},"publicIp":{"type":"string","description":"public IP address of the host","nullable":true},"provisionedHostname":{"type":"string","description":"host name provisioned by the system","nullable":true},"operationalStatus":{"type":"string","description":"operational status of the host","nullable":true},"poweredOn":{"type":"boolean","description":"true if the host is powered on"},"provisioningState":{"type":"string","description":"provisioning status of the host","nullable":true}},"additionalProperties":false}}}} ``` # Images ## GET /api/v1/servers/images/GetOperatingSystemImages > Get a list of operating sytem images available for the tenant ```json {"openapi":"3.0.4","info":{"title":"DenvrCloud API","version":"v1"},"tags":[{"name":"servers/images"}],"paths":{"/api/v1/servers/images/GetOperatingSystemImages":{"get":{"tags":["servers/images"],"summary":"Get a list of operating sytem images available for the tenant","operationId":"ApiV1ServersImagesGetoperatingsystemimagesGet","responses":{"200":{"description":"OK","content":{"text/plain":{"schema":{"$ref":"#/components/schemas/ListResultDtoOfOperatingSystemImage"}},"application/json":{"schema":{"$ref":"#/components/schemas/ListResultDtoOfOperatingSystemImage"}},"text/json":{"schema":{"$ref":"#/components/schemas/ListResultDtoOfOperatingSystemImage"}}}},"400":{"description":"Bad Request"},"401":{"description":"Unauthorized"},"403":{"description":"Forbidden"},"500":{"description":"Internal Server Error"}}}}},"components":{"schemas":{"ListResultDtoOfOperatingSystemImage":{"type":"object","properties":{"items":{"type":"array","items":{"$ref":"#/components/schemas/OperatingSystemImage"},"nullable":true}},"additionalProperties":false},"OperatingSystemImage":{"type":"object","properties":{"id":{"type":"integer","format":"int32"},"name":{"type":"string","nullable":true},"distribution":{"type":"string","nullable":true},"version":{"type":"string","nullable":true}},"additionalProperties":false}}}} ``` # Snapshots ## GET /api/v1/servers/snapshots/GetSnapshots > Get list of snapshots. ```json {"openapi":"3.0.4","info":{"title":"DenvrCloud API","version":"v1"},"tags":[{"name":"servers/snapshots"}],"paths":{"/api/v1/servers/snapshots/GetSnapshots":{"get":{"tags":["servers/snapshots"],"summary":"Get list of snapshots.","operationId":"ApiV1ServersSnapshotsGetsnapshotsGet","parameters":[{"name":"Cluster","in":"query","schema":{"type":"string"}}],"responses":{"200":{"description":"OK","content":{"text/plain":{"schema":{"$ref":"#/components/schemas/ListResultDtoOfSnapshotDetailsItem"}},"application/json":{"schema":{"$ref":"#/components/schemas/ListResultDtoOfSnapshotDetailsItem"}},"text/json":{"schema":{"$ref":"#/components/schemas/ListResultDtoOfSnapshotDetailsItem"}}}},"400":{"description":"Bad Request"},"401":{"description":"Unauthorized"},"403":{"description":"Forbidden"},"500":{"description":"Internal Server Error"}}}}},"components":{"schemas":{"ListResultDtoOfSnapshotDetailsItem":{"type":"object","properties":{"items":{"type":"array","items":{"$ref":"#/components/schemas/SnapshotDetailsItem"},"nullable":true}},"additionalProperties":false},"SnapshotDetailsItem":{"type":"object","properties":{"id":{"type":"string","nullable":true},"namespace":{"type":"string","nullable":true},"sourceName":{"type":"string","nullable":true},"osImage":{"type":"string","nullable":true},"customPackage":{"type":"string","nullable":true},"rootDiskSize":{"type":"string","nullable":true},"creationDate":{"type":"string","format":"date-time"},"username":{"type":"string","nullable":true},"tenancyName":{"type":"string","nullable":true},"readyToUse":{"type":"boolean"}},"additionalProperties":false}}}} ``` ## GET /api/v1/servers/snapshots/GetSnapshot > Get detailed information on specific snapshot. ```json {"openapi":"3.0.4","info":{"title":"DenvrCloud API","version":"v1"},"tags":[{"name":"servers/snapshots"}],"paths":{"/api/v1/servers/snapshots/GetSnapshot":{"get":{"tags":["servers/snapshots"],"summary":"Get detailed information on specific snapshot.","operationId":"ApiV1ServersSnapshotsGetsnapshotGet","parameters":[{"name":"Id","in":"query","description":"Name of snapshot","required":true,"schema":{"type":"string"}},{"name":"Namespace","in":"query","description":"The namespace/vpc.","required":true,"schema":{"type":"string"}},{"name":"Cluster","in":"query","description":"The cluster you're operating on","required":true,"schema":{"type":"string"}}],"responses":{"200":{"description":"OK","content":{"text/plain":{"schema":{"$ref":"#/components/schemas/SnapshotDetailsItem"}},"application/json":{"schema":{"$ref":"#/components/schemas/SnapshotDetailsItem"}},"text/json":{"schema":{"$ref":"#/components/schemas/SnapshotDetailsItem"}}}},"400":{"description":"Bad Request"},"401":{"description":"Unauthorized"},"403":{"description":"Forbidden"},"500":{"description":"Internal Server Error"}}}}},"components":{"schemas":{"SnapshotDetailsItem":{"type":"object","properties":{"id":{"type":"string","nullable":true},"namespace":{"type":"string","nullable":true},"sourceName":{"type":"string","nullable":true},"osImage":{"type":"string","nullable":true},"customPackage":{"type":"string","nullable":true},"rootDiskSize":{"type":"string","nullable":true},"creationDate":{"type":"string","format":"date-time"},"username":{"type":"string","nullable":true},"tenancyName":{"type":"string","nullable":true},"readyToUse":{"type":"boolean"}},"additionalProperties":false}}}} ``` ## POST /api/v1/servers/snapshots/CreateSnapshot > Create a new snapshot from an existing virtual machine. ```json {"openapi":"3.0.4","info":{"title":"DenvrCloud API","version":"v1"},"tags":[{"name":"servers/snapshots"}],"paths":{"/api/v1/servers/snapshots/CreateSnapshot":{"post":{"tags":["servers/snapshots"],"summary":"Create a new snapshot from an existing virtual machine.","operationId":"ApiV1ServersSnapshotsCreatesnapshotPost","requestBody":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/CreateSnapshotInput"}},"text/json":{"schema":{"$ref":"#/components/schemas/CreateSnapshotInput"}},"application/*+json":{"schema":{"$ref":"#/components/schemas/CreateSnapshotInput"}}}},"responses":{"202":{"description":"Accepted","content":{"text/plain":{"schema":{"$ref":"#/components/schemas/SnapshotDetailsItem"}},"application/json":{"schema":{"$ref":"#/components/schemas/SnapshotDetailsItem"}},"text/json":{"schema":{"$ref":"#/components/schemas/SnapshotDetailsItem"}}}},"400":{"description":"Bad Request"},"401":{"description":"Unauthorized"},"403":{"description":"Forbidden"},"500":{"description":"Internal Server Error"},"503":{"description":"Service Unavailable"}}}}},"components":{"schemas":{"CreateSnapshotInput":{"required":["cluster","namespace"],"type":"object","properties":{"name":{"type":"string","nullable":true},"namespace":{"minLength":1,"type":"string"},"cluster":{"minLength":1,"type":"string"},"sourceVMName":{"type":"string","nullable":true}},"additionalProperties":false},"SnapshotDetailsItem":{"type":"object","properties":{"id":{"type":"string","nullable":true},"namespace":{"type":"string","nullable":true},"sourceName":{"type":"string","nullable":true},"osImage":{"type":"string","nullable":true},"customPackage":{"type":"string","nullable":true},"rootDiskSize":{"type":"string","nullable":true},"creationDate":{"type":"string","format":"date-time"},"username":{"type":"string","nullable":true},"tenancyName":{"type":"string","nullable":true},"readyToUse":{"type":"boolean"}},"additionalProperties":false}}}} ``` ## DELETE /api/v1/servers/snapshots/DeleteSnapshot > Delete snapshot. ```json {"openapi":"3.0.4","info":{"title":"DenvrCloud API","version":"v1"},"tags":[{"name":"servers/snapshots"}],"paths":{"/api/v1/servers/snapshots/DeleteSnapshot":{"delete":{"tags":["servers/snapshots"],"summary":"Delete snapshot.","operationId":"ApiV1ServersSnapshotsDeletesnapshotDelete","parameters":[{"name":"Id","in":"query","description":"Name of snapshot","required":true,"schema":{"type":"string"}},{"name":"Namespace","in":"query","description":"The namespace/vpc.","required":true,"schema":{"type":"string"}},{"name":"Cluster","in":"query","description":"The cluster you're operating on","required":true,"schema":{"type":"string"}}],"responses":{"202":{"description":"Accepted","content":{"text/plain":{"schema":{"$ref":"#/components/schemas/SnapshotCommandOutput"}},"application/json":{"schema":{"$ref":"#/components/schemas/SnapshotCommandOutput"}},"text/json":{"schema":{"$ref":"#/components/schemas/SnapshotCommandOutput"}}}},"400":{"description":"Bad Request"},"401":{"description":"Unauthorized"},"403":{"description":"Forbidden"},"500":{"description":"Internal Server Error"}}}}},"components":{"schemas":{"SnapshotCommandOutput":{"type":"object","properties":{"id":{"type":"string","nullable":true}},"additionalProperties":false}}}} ``` # FAQs # Desktop vs data center GPUs Why NVIDIA A100/H100 instead of workstation GPUs? The NVIDIA family of data center GPUs (A100/H100/B100) provide significant advantages over all classes of desktop GPUs: ## Tensor Float (TF32) data type NVIDIA A100 introduces a new hardware data type called 'TF32' which speeds single-precision work, while maintaining accuracy and using no new code. TF32 is a hybrid data type that delivers sufficient precision for tensor operations without the full size of FP32 that slows processing and bloats memory. ![](https://blogs.nvidia.com/wp-content/uploads/2020/05/tf32-Mantissa-chart-hi-res-FINAL-400x255.png) The result is an **8X speed improvement** over FP32 where the reduced precision still matches FP16. This is optimal for matrix math used in Machine Learning. Read more: * ## Structural sparsity Sparsity is an optimization technique that creates a more efficient network from a dense network, and can make better predictions in a limited time budget, react more quickly to unexpected input, or fit into constrained deployment environments. ![](https://developer-blogs.nvidia.com/wp-content/uploads/2021/07/inference-sparsity-ampere-tensorRT.png) Use of sparsity for inferencing achieves **2X performance improvement** for TF32, FP16, and INT8 operations over non-sparse operations. Read more: * ## Double-precision floating (FP64) IEEE 754 double-precision binary floating-point format provides 16 decimal digits of precision. This is format is used for scientific computations with strong precision requirements. Most GPUs have severely limited FP64 performance - operating at 1/32 of FP32 performance. The A100 computes FP64 at 1/2 of FP32 which is a dramatic acceleration over CPU, desktop GPUs, and other data center GPUs. ![](https://miro.medium.com/max/1236/0*HSK9nCIyoPm7CpBY.png) Read more: * ## NvLink for advanced multi-GPU HGX A100 systems include NVIDIA Gen4 NvLink (SXM4) which provides 600 GB/s of GPU-to-GPU bandwidth. This is an over 8X improvement over PCIe Gen4 which offers 64 GB/s of bandwidth. The HGX H100 systems use Gen5 NvLink (SXM5) for 900 GB/s. This architecture is critical for models that require more GPU working memory than is available in a single GPU.
NVLink and NVSwitch Work Together
## Multi-Instance GPU (MIG) MIG partitions NVIDIA GPUs into as many as seven instances, each hardware isolated with its own high-bandwidth memory, cache, and compute cores.\ \ These 'fractional' GPU configurations provide 10GB or 20GB memory options. The benefits of this are: 1. Allocate only GPU resources needed for a workload 2. Test GPU features (per above) without needing to configure a full system 3. Save costs for education and analytics tasks before operating with full GPU and multi-GPU configurations. Read more: * ## Compute Fabric AI Factory deployments include dedicated networks exclusively for the use by GPUs and are capable of scaling from hundreds to thousands of nodes. Denvr Cloud implements fabrics from 800G to 3200G per node for AI Training using either InfiniBand or RoCE v2 technologies. Multi-node is mandatory for Large Language Models (LLM) training like GPT-3 that are unachievable with Ethernet-based clusters.
Read more: * # Differences of bare metal and virtual machines ## Bare metal Bare metal clusters provide direct access to physical hardware for greater control and customization of the environment. This includes but is not limited to: * physical CPUs, not virtualized CPUs * no hypervisor overhead which requires host CPU and memory resources * ability to change symmetric multi threading (SMT) and NUMA BIOS settings * full access to local NMVe block devices to configure filesystems and RAID * enhanced security by removing the hypervisor layer Denvr Cloud has limited observability into bare metal hosts which which requires additional responsibility of the tenant to monitor and report system issues for analysis. ## Virtual machines Virtual machines provide access to GPUs and compute resources with a minor overhead required for the hypervisor. The primary benefits of virtualization, even for full nodes, are: * choice of operating system images to use during provisioning * 1-5 minutes to boot the instance (depends primarily on number of vCPUs assigned) * ability to resize allocated resources (CPU, memory disks) as requirements change * less likelihood of over-provisioning resources versus large bare metal hosts * usage of snapshots for backups and starting new instances * self-service management via Denvr Cloud console and APIs {% hint style="success" %} GPU processing and InfiniBand/RoCE fabrics are not impacted by virtualization as the devices are passed-through to the guest O/S. This provides the same level of performance, control, and isolation benefits as a bare metal host. {% endhint %} ## Feature comparison ### Performance
Bare MetalVirtual Machines
Overhead✅ NoneHypervisor requires CPU and memory resources
Consistency✅ High consistency in performancePotential variability in performance
### Security
Bare MetalVirtual Machines
Tenant isolation✔️ Single tenant only✔️ Single or multi-tenant
Multi-user accessMay require additional software to manage user isolation and Quality of Service✅ Quality of service is enforced
Security risk✅ Lowest risk of cross-tenant impactHigher risk due to shared infrastructure. Virtualization is a mature technology in use since the early 2000's.
Data privacy✔️ Denvr has no mechanism to access the system, including for operational support✔️ Denvr can monitor and maintain the system hardware instead of the user
### Management
Bare MetalVirtual Machines
Node types✔️ Full node only✔️ Allows customization of machine resources including single-GPU instances
ScalabilityFull node only✅ Instance sizes can be changed dynamically without re-provisioning
Software and driversDenvr only installs the GPU and fabric drivers.✅ Pre-configured images are available with required software dependencies.
Time to launchSlower✅ Fastest
Backup and restoreDifficult to backup the operating system for recovery✅ Simple to snapshot, clone, copy, and duplicate machine images.
# GPU monitoring Monitoring GPU performance is critical to ensure maximum use of the hardware, and to identify bottlenecks and issues. ## JupyterLab NVDashboard NVDashboard is a JupyterLab extension for displaying NVIDIA GPU usage dashboards. It enables users to visualize system hardware metrics within the same interactive environment they use for development and data analysis. Supported metrics include: * GPU compute utilization * GPU memory consumption * PCIe throughput * NvLink throughput ### Installation Select Terminal from JuptyterLab
Install the package from PyPI ``` pip install jupyterlab_nvdashboard ``` Restart the JupyterLab application from the Denvr AI Cloud Dashboard to enable the package. Read more: * ## Linux command-line NVIDIA drivers are preinstalled with the `nvidia-smi` command which can be used to find performance metrics of the GPUs. This example shows a single A100 MIG 10 GB instance.
You can get a continuous output of this view by running: ``` nvidia-smi -l ``` Another technique is to use the `watch` command to run `nvidia-smi` every X seconds, clearing the screen each time before displaying new output: ``` watch -n 5 nvidia-smi ``` The `nvidia-smi dmon` command can also stream output in tabular format which is more easily used for logging and automated monitoring.
The Linux man page has additional information how to configure `nvidia-smi dmon` to select specific output columns and to provide CSV-style formatting.
## Promethesus exporter NVIDIA provides a tool called DGCM-Exporter which streams metrics into Promethesus. This code should be installed as a service by system administrators depending on your deployment preferences (Kubernetes, docker, local install). ``` $ docker run -d --gpus all --rm -p 9400:9400 nvcr.io/nvidia/k8s/dcgm-exporter:3.3.5-3.4.0-ubuntu22.04 ``` You can verify metrics as follows: ``` $ curl localhost:9400/metrics ``` Read more: * # Using Github with SSH keys GitHub removed support for Password Authentication while using HTTPS protocol in August 2021. It is convenient (and recommended) to access repos securely using SSH keys. ## Generate a public/private RSA key pair Skip to next step if you have an existing SSH key pair. {% hint style="info" %} Windows users can use either PowerShell, Windows Terminal, or WSL (Windows Subsystem for Linux). Mac and Linux users should use their preferred terminal. {% endhint %} Run the `ssh-keygen` command with default arguments to generate and replace your private keypair. ``` ssh-keygen -t rsa ``` You will be prompted to provide a filename and an additional password if required. {% hint style="info" %} It is common to NOT use a password with the private key because the operating system already controls user access. {% endhint %} The output will look like this:
## Copy the public key Use the terminal to display the public key, highlight, and copy to clipboard. {% hint style="danger" %} The public key will have a .pub file extension. You should never share your private key! {% endhint %} ``` $ cd $HOME/.ssh $ cat id_rsa.pub ``` The output will look similar to the following:
## Save the public key to GitHub Navigate to 'Settings' under your GitHub profile. In the Left navigation select 'SSH and GPG keys':
Select 'New SSH key', add a title with any descriptive name, and paste the public key into the Key textarea:
Your repos will now show clone commands using SSH:
# Data persistence and recovery {% hint style="danger" %} Data on root file systems and local storage are not persisted after instance termination and cannot be recovered by Denvr Support. This applies to Virtual Machines as well as Bare Metal. {% endhint %} ## Virtual Machines Machine root disks are held on protected storage while an instance is Running or in Stopped state. This is slightly different than local scratch storage which exists on the physical host and is wiped during Stopped state. ## Bare Metal Machine root disks are deployed directly onto the assigned host. The host hardware uses RAID-1 protected boot disks for durability and is retained across host reboot. Local scratch storage is not affected by node reboot. It is recommended that users configure local block storage using with RAID-1 to protect data, while still gaining the parallel throughput advantage of RAID-0 striping. This can be setup using `ZFS``, mdraid` and similar Linux systems. Host boot disks and local storage is wiped when reservations are complete. ## High-Performance File Storage Users should use High-Performance File Storage for data that persists beyond the life cycle of the machine instance. This is implemented via NFS and is available to Applications, Virtual Machines, and Bare Metal. NFS can be mounted by user after a Virtual Machine or Bare Metal host are created. File storage is not replicated between Regions and is not backed for recovery. The user must assess their own data availability requirements in the unlikely scenario that Denvr Cloud is unavailable. # Do you support Kubernetes? We do not currently offer Managed Kubernetes. Users can install and operate K8s themselves on VMs and Bare Metal, for example using [CNCF Kubernetes](https://kubernetes.io/releases/download/) and [MicroK8s](https://microk8s.io/). Containers can be run in Denvr Cloud by creating a Custom Application. This is similar to managed Kubernetes in that user applications are monitored for failover and recovery as well as supporting dynamic pod changes. The limitation is that you cannot install Kubernetes packages or use the kubectl command-line and APIs. # Installing GPU drivers Operating systems for VMs and Bare Metal are pre-installed with GPU drivers based on vendor recommended versions and dependencies. * NVIDIA CUDA (GPU libraries) * NVIDIA Fabric Manager (for NVLink-enabled instances) * NVIDIA InfiniBand (OFED) Applications are containers provided by vendors and do not require any further setup. # What is the network bandwidth? {% hint style="info" %} Internet bandwidth depends on many factors including ISP speed, use of multiple connections, serving capacity, and receiving bandwidth. {% endhint %} Denvr Cloud does not throttle or limit customer traffic except for fair use of our total capacity shared in a region. Internet access is a shared resource for each cluster. Denvr actively monitors and plans for capacity growth as needed by clients. Please contact Denvr Support if you have specific bandwidth requirements. ## MSC1 (Calgary, Canada)
NetworkHost Bandwidth
Cluster network (between nodes)Up to 25 Gbps
Compute network (InfiniBand)800G InfiniBand
Storage accessUp to 25 Gbps
Internet access10Gbps (shared by region)
## HOU1 (Houston, USA) | Network | Host Bandwidth | | ------------------------------- | --------------------------- | | Cluster network (between nodes) | Up to 200 Gbps | | Compute network (InfiniBand) | 1600G and 3200G InfiniBand | | Storage access | Up to 200 Gbps | | Internet access | 100 Gbps (shared by region) | # What ports are publicly accessible? Every client Application, Virtual Machine and Bare Metal host are assigned a unique Public IP address. There are no ports or services blocked at the network layer. Users should take best efforts to protect their instance including, for example, use of host-based firewalls ([ufw](https://wiki.ubuntu.com/UncomplicatedFirewall)) and intrustion prevention software ([fail2ban](https://github.com/fail2ban/fail2ban)). {% hint style="info" %} Network Firewall Services are under development for a future release of Denvr Cloud {% endhint %} # What is persistent local storage?
As discussed in the [Storage](/docs/platform/storage) docs, our virtual machines come with up to 30 TB of local direct attached storage (DAS) . This local storage is ideal for caching model checkpoints and training datasets to minimize IO bottlenecks. For on-demand usage, DAS volumes are always ephemeral. Users are responsible for formatting, mounting, and syncing data to these volumes. DAS volumes are erased when virtual machines are stopped, requiring any valuable data to be backed up to our network attached storage (NAS) volumes. \ \ For reserved capacity, users can configure persistent direct attached storage (PDAS) and optionally request the volume be automatically formatted and mounted. The data on these volumes will be preserved across instances stops and restarts. However, this will also pin your virtual machine to the reserved host machine where the DAS is located. # Adding DAS to /etc/fstab You may want to automatically mount your DAS during reboots (e.g., \`reboot\`, \`shutdown -r\`, \`systemctl reboot\`). In these cases, the host will not be released and your DAS will remain intact. However, for safety purposes we recommend using the following \`/etc/fstab\` entry. 
# device    mount                           fs-type      options                                       dump pass
/dev/vdc    /home/ubuntu/direct-attached    ext4         defaults,nofail,x-systemd.device-timeout=5    0    2
#### Explanation * device -\`/dev/vdc\` * mount - \`/home/ubuntu/direct-attached\` (example) * fs-type - ext4 (example) * options: * defaults - start with the filesystem defaults * **no-fail** - DAS volume cannot block system boot * **x-systemd.device-timeout=5** - only wait 5s on filesystem during boot * dump (0) - do not run backup * pass (2) - run fsck after boot The **\`no-fail\`** and **\`x-systemd.device-timeout=5\`** options are particularly important since stopping and restarting a VM on another host would cause mounting the DAS to fail and block the VM boot process. Make sure not to set the timeout to 0, as this translates to infinite timeout. #### Tip To validate your \`/etc/fstab\` changes run a \`mount -a\` before trying a reboot. # Provisioning States Virtual machine and application lifecycles explained Across various Denvr services, resources will transition between multiple states during their lifecycle. In particular, applications and virtual machines can have the following status values: * PENDING - waiting to transition to another state * PENDING\_RESOURCES - Denvr is allocating resources * PENDING\_READINESS - resources have been allocated and we are waiting your application or virtual machine to come online. * ONLINE - your application or virtual machine has been provisioned and is online * OFFLINE - the resource has been stopped, but still exists * FAILED - an error has occurred, please contact support These status codes are available from both the console UI or REST API. Apart from persisted root volumes and additional storage, you are only billed during the PENDING\_READINESS or ONLINE states. ### How does PENDING\_READINESS work? Virtual machines and applications take time to boot after resources have been allocated. Our readiness check involves pinging port 22 on virtual machines, or a user defined readiness port for applications. How long a virtual machine or application takes to pass the readiness is dependent on your particular VM or application startup process. On our base images, this should only a few minutes. Common causes of long (or infinite) pending readiness periods include: * Corrupted \`/etc/fstab\` files which block the boot process. See ["Adding DAS to /etc/fstab"](/docs/additional-information/faqs/adding-das-to-etc-fstab). * Incorrectly configured firewall (\`ufw\`) blocking ssh access * Addition of slow or failing init scripts * Applications that pull down large models or datasets prior to starting the service # What are Resource Pools? This page explains differences, use cases, and how to choose the right resource pool based on customer's needs. Resource Pools are logical groupings of compute and GPU resources within Denvr’s infrastructure. They determine how workloads are scheduled and what level of availability and performance guarantees apply. There are two main types of resource pools: * **On-Demand Resource Pools** * **Reserved Resource Pools** Each is designed to serve different workload types and customer requirements. *** ### On-Demand Resource Pools #### Overview An **on-demand resource pool** is a shared pool of compute and GPU resources available on a best-effort basis. VMs scheduled in this pool are provisioned only when the requested configuration (e.g., specific GPU, memory, CPU) becomes available. This model is ideal for flexible, cost-effective usage, where resource availability is not time-critical. #### Key Characteristics * **Shared Access**: The pool is shared across multiple tenants and workloads. * **Dynamic Allocation**: Resources are allocated only if available at the time of the request. * **No Availability Guarantee**: Resource allocation is not guaranteed. VMs may remain in a pending state if the required configuration is not currently available. * **First-Come, First-Served**: Scheduling is non-prioritized and based solely on real-time availability. #### Common Use Cases * Development or testing environments * Batch or burst workloads with no strict SLA * Non-critical jobs that can tolerate delays #### Limitations * VMs may become **unschedulable** during peak usage or when demand for certain configurations (e.g., A100 80GB GPUs) exceeds current capacity. * Users may see errors such as `0/45 nodes are available: 45 Insufficient nvidia.com/gpu` on our dashboard when creating VMs. *** ### Reserved Resource Pools #### Overview A **reserved resource pool** is a dedicated allocation of compute and GPU resources set aside specifically for a customer or workload. These resources are provisioned and guaranteed in advance, providing predictable availability and performance. #### Key Characteristics * **Guaranteed Availability**: Resources are reserved and pre-allocated to your organization. * **Single-Tenant or Shared-Internal Use**: Pools can be configured for single-customer isolation or shared use across internal workloads. * **Consistent Performance**: Reserved capacity ensures consistent scheduling and uptime, especially for production workloads. * **Custom Configuration**: Customers can specify GPU type, memory, CPU, and network requirements tailored to their needs. #### Common Use Cases * Production workloads with strict SLA requirements * Workloads requiring consistent and uninterrupted access to specific hardware configurations * Multi-VM environments with dependencies between nodes * AI/ML training pipelines requiring large contiguous GPU allocations #### Onboarding Process To set up a reserved pool: 1. Contact your **Account Manager** or our [**Customer Experience team**](https://www.denvrdata.com/contact-our-team). 2. We will assist in: * Evaluating your workload profile * Designing and sizing the reserved pool * Provisioning dedicated infrastructure 3. Access controls and tenancy setup will be handled as part of the onboarding. # How does sudo and passwordless sudo work on my VM? By default, virtual machines provisioned on our platform come with a user account - ubuntu - that has sudo privileges. This means you can run administrative commands prefixed with sudo to execute them as the root user. #### What is \`sudo\`? * sudo stands for “superuser do.” * It allows a permitted user to execute commands as the superuser or another user, as specified by the security policy. * For example: ``` sudo apt update sudo systemctl restart nginx ``` #### What is passwordless \`sudo\`? In many base images, passwordless sudo is enabled, meaning: * You do not need to enter your password when running sudo commands. * This is configured in the image by adding the user to the sudoers file with the NOPASSWD directive, such as: ``` user ALL=(ALL) NOPASSWD:ALL ``` This setup is useful for automation, scripts, and quick administrative access without interactive prompts. #### Does Denvr set passwords or manage sudo configuration? No. Denvr does not set user passwords during VM provisioning, nor do we install or run any agents within your virtual machines. Your VM is launched exactly as defined in your chosen OS image, with: * No changes to sudo configuration or password settings by Denvr. * No agents installed to manage user accounts or system settings. All password configurations, sudo policies, and user management remain entirely under your control. #### How does this impact security? While passwordless sudo improves usability, it grants full administrative rights without authentication prompts. Therefore: * Ensure your SSH keys are secured and only authorized personnel have access to your VMs. * If your security or compliance policies require password prompts for sudo, modify the sudoers configuration within your VM to enforce that as needed. # How are my VMs and Applications billed? How Denvr bills for compute instances —Virtual Machines (VMs) and Applications (Apps)—that are stopped or idle. The details below reflect current platform behavior ### What happens when I stop a Virtual Machine or Applications from the Denvr console? When you stop a VM or App via the Denvr console: * All compute resources and ephemeral storage are deallocated. * Persisted storage continues to incur charges until explicitly removed by the user. *** ### Does stopping a Virtual Machine or Application from inside the OS (e.g., using `shutdown`) stop billing? No - stopping from within the guest OS **does not** deallocate compute resources. The instance remains in an active, billable state. You must stop it using the **Denvr console** or APIs to halt compute billing. *** ### What are the differences in billing between VMs and Apps when stopped? | Feature | Virtual Machines (VMs) | Applications (Apps) | | ----------------------------- | --------------------------------------------------- | ------------------------------------ | | Stoppable from Console | Yes | Yes | | Compute charges while stopped | No | No | | Root storage | Persistent root disk (billed) | Ephemeral root\_FS (not billed) | | Filesystem storage | Billed until data is removed | Billed until data is removed | | Storage billed while stopped | Yes (root disk and filesystems) | Yes (filesystems only) | | Storage deletion behavior | Root disk deleted on VM delete; filesystems persist | Filesystems persist after App delete | *** ### What storage types are billed when a Virtual Machine is stopped? | Storage Type | Billed While Running | Billed While Stopped | Deleted When | | ----------------- | -------------------- | -------------------- | ---------------------------------------- | | Root Disk | Yes | Yes | When the VM is deleted | | Filesystems | Yes | Yes | When user deletes data inside filesystem | | Ephemeral Storage | No | No | Discarded on stop | *** ### What storage types are billed when an Application is stopped? | Storage Type | Billed While Running | Billed While Stopped | Deleted When | | -------------------- | -------------------- | -------------------- | ---------------------------------------- | | Root\_FS (ephemeral) | Included in compute | Not applicable | Discarded on stop | | Filesystems | Yes | Yes | When user deletes data inside filesystem | | Ephemeral Storage | No | No | Discarded on stop | *** ### If I delete a Virtual Machine or Application, will all storage charges stop? Not necessarily. * For **VMs**, deleting the instance also deletes the **root disk**, but any attached **filesystems persist** and continue to incur charges unless the user deletes the data within them. * For **Apps**, the **ephemeral root\_FS** is discarded automatically, but **filesystems persist** and continue to be billed until emptied. *** ### How do I stop being billed for persisted storage? You must **delete the data** within any mounted filesystems. Simply deleting the VM or App will not remove or unbill filesystem storage. *** ### Can I snapshot or back up my data? Not at this time.\ \ Denvr does **not** provide a user-accessible way to snapshot or back up data. If data is deleted, it cannot be recovered. Although the platform performs internal storage replication for fault tolerance within a failure domain, this is not user-controlled and does not affect billing. *** ### Where can I monitor my usage and costs? Use the **Cost and Usage → Cost Explorer** screen in the Denvr console to track your current charges.
Note: **Reserved compute charges** are billed manually and are **not yet visible** in Cost Explorer. Full cost integration is planned in a future release. *** ### Summary * Use the **Denvr console** to stop VMs or Apps in order to deallocate compute and ephemeral storage. * **Persisted storage** is the only component billed while an instance is stopped. * **VM root disks** are deleted when the VM is deleted. * **Filesystems** persist after instance deletion and must be manually cleared to stop billing. * No backup or snapshot capability is currently available to users. # Policies # Terms of Use # Shared responsibility model Security is a shared responsibility between Denvr and the customer. Both parties need to collaborate and understand their respective roles to ensure that all security aspects are handled appropriately. For example, while Denvr AI Cloud takes full responsibility for hosting physical infrastructure and ensuring network security, users are responsible for managing client-side encryption, open internet ports, and offsite backup copies of data. ## Denvr AI Cloud's Responsibilities
FunctionDescription
Physical security
  • Protecting the physical servers, storage, and networking equipment within our data centers.
  • This includes access controls, monitoring, environmental safeguards, and other measures that ensure the hardware is safe from unauthorized access or damage.
Infrastructure security
  • The security of the physical infrastructure, which includes the compute, storage, and network resources used to deliver our infrastructure platform.
  • Security patches and updates for the hardware and hypervisors.
Virtualization and network isolation
  • We ensure the security of the virtualized environment, including the hypervisors and orchestration tools that manage virtual machines (VMs) and containers.
  • Network segmentation and isolation between tenants to prevent any risk of unauthorized access between accounts.
Service availability
  • We ensure that the platform remains available by implementing redundant systems, monitoring, and incident response capabilities.
  • This includes automatic failover systems and backup mechanisms to protect against hardware failures.
Compliance certifications
  • Denvr maintains compliance with SOC 2 and GDPR, and others as applicable.
## Customer's Responsibilities
FunctionDescription
Identity and Access Management
  • Customers are responsible for managing access to their cloud environment.
  • This involves creating user accounts, assigning permissions, enabling multi-factor authentication (MFA), and securing shell access to instances.
Application Security
  • Customers must ensure that the applications they deploy are secure. This includes regular security testing, code reviews, vulnerability scanning, and patch management
Operating System
  • Keeping operating systems up to date with the latest security patches.
  • Installing and configuring anti-malware and firewall software.
  • Managing software dependencies and ensuring third-party libraries or frameworks are secure.
Backup and Disaster Recovery
  • While we provide infrastructure-level redundancy, customers must ensure their own backup and recovery plans are in place for the data and applications they control.
# Maintenance policy Denvr AI Services hardware maintenance requirements To ensure performance, reliability and enhancement of our AI cloud platform, we have implemented the following guidelines for managing both planned and unplanned maintenance activities. These guidelines form an integral part of our platform's service commitments and are incorporated into our Terms of Service (TOS). Periodic issues and maintenance should be expected as AI Factory hardware has higher expected failure rates versus traditional CPU products. ## Scheduled Maintenance Scheduled maintenance refers to mandatory activities to upgrade, patch, and ensure availability of our AI systems. This may include device firmware, drivers, and hypervisor upgrades. These activities are communicated in advance with tenant administrators to minimize client disruption. Changes can be temporarily deferred as required but only for 1-2 months pending the criticality of the upgrade requirement. * Notifications provided by email and in-app dashboard * Maintenance windows to be preferred in off-peak hours * Expected downtime will be communicated * Return to production will be coordinated with the tenant ## Emergency Maintenance Some situations may require emergency maintenance, for example critical security vulnerabilities (CVE) and in response to system failures including hardware failures (GPU drops, transceivers and fibre link quality, etc). * Notifications will be provided even on short notice * Maintenance to be coordinated with the tenant if required. Clients may choose to accept degraded performance if preferred to have a less impacting change windows * Denvr will provide a summary of what was fixed, root cause analysis (RCA), and any follow up actions. ## Bare Metal Maintenance Denvr may detect system issues via network monitoring and BMC management. Because we have no access to bare metal nodes, we may require the tenant to assist with diagnostic commands and output to ensure system health. Clients may choose to defer diagnostics and accept existing or potential degrades in order to minimize impact. However diagnostics and change windows may still be required per Scheduled Maintenance requirements.