Skip to main content

File Stores & Persistent Storage

Attach persistent storage to your assets for stateful workloads like databases, file uploads, and caches. Codiac integrates with cloud storage (AWS S3, Azure Blob) and Kubernetes persistent volumes.

What Are File Stores?​

File stores provide persistent storage that survives pod restarts and can be shared across multiple pods. Unlike ephemeral container storage that's deleted when a pod terminates, file store data persists independently.

Business Value:

  • Data persistence: Databases, uploads, and user data survive deployments and failures
  • Scalability: Share storage across multiple pod replicas
  • Cloud integration: Leverage managed storage (S3, Azure Blob) without managing infrastructure
  • Backup & recovery: Cloud storage includes automatic backups and disaster recovery

Storage Types​

TypeDescriptionBest ForExamples
Kubernetes VolumesBlock storage attached to pods as mounted directoriesDatabases, logs, temporary filesPostgreSQL data directory, MongoDB storage
Cloud Object StorageScalable blob/object storage (S3, Azure Blob)File uploads, backups, mediaUser uploads, document storage, images
Shared File SystemsNetwork file systems accessible by multiple podsShared assets, static contentShared uploads, CMS files, assets

How File Stores Work​

Asset (Container)
      ↓
  [Volume Mount]
      ↓
Persistent Volume Claim (PVC)
      ↓
Persistent Volume (PV)
      ↓
Cloud Storage (AWS EBS, Azure Disk, S3, etc.)

Workflow:

  1. Create file store definition (captures cloud storage configuration)
  2. Attach volume to asset (mount path inside container)
  3. Codiac creates PersistentVolumeClaim (PVC)
  4. Kubernetes binds PVC to cloud storage
  5. Container accesses storage at mount path

Managing File Stores with CLI​

Capturing File Store Configuration​

Import existing cloud storage resources into Codiac:

cod filestore capture \
  --provider aws \
  --name my-s3-bucket \
  --enterprise my-company

Expected Outcome:

  • Codiac fetches S3 bucket configuration from AWS
  • File store definition saved in enterprise
  • Can now be attached to assets as volumes

Supported Providers:

  • aws - AWS S3, EBS
  • azure - Azure Blob Storage, Azure Disk
  • gcp - Google Cloud Storage (GCS)

Creating a Volume for an Asset​

Attach persistent storage to an asset:

cod asset volume create /data \
  --asset postgres-db \
  --cabinet prod \
  --size 100Gi \
  --storage-class standard

Parameters:

  • /data - Mount path inside the container
  • --size - Volume size (e.g., 10Gi, 100Gi, 1Ti)
  • --storage-class - Storage tier (standard, ssd, premium)

Expected Outcome:

  • PersistentVolumeClaim created with 100Gi capacity
  • Volume mounted at /data inside postgres container
  • Data in /data persists across pod restarts

Example: Database with Persistent Storage​

PostgreSQL with 500GB volume for data directory:

cod asset volume create /var/lib/postgresql/data \
  --asset postgres \
  --cabinet prod \
  --size 500Gi \
  --storage-class ssd

Result:

  • PostgreSQL stores database files in /var/lib/postgresql/data
  • Data persists across deployments, scaling, and pod failures
  • Uses SSD-backed storage for better I/O performance

Example: File Upload Service​

Node.js API with volume for user file uploads:

cod asset volume create /app/uploads \
  --asset api-server \
  --cabinet prod \
  --size 200Gi \
  --storage-class standard

Application Code:

// Files saved to /app/uploads are persisted
const uploadPath = '/app/uploads';
app.post('/upload', upload.single('file'), (req, res) => {
  const filePath = path.join(uploadPath, req.file.filename);
  // File persists even if container restarts
});

Viewing Attached Volumes​

List volumes attached to an asset:

cod asset view postgres --cabinet prod

Expected Output:

Asset: postgres
Cabinet: prod
Volumes:
  - Mount Path: /var/lib/postgresql/data
    Size: 500Gi
    Storage Class: ssd
    Status: Bound

Deleting a Volume​

Remove volume from asset (data is retained in cloud storage):

cod asset volume delete /var/lib/postgresql/data \
  --asset postgres \
  --cabinet prod

Warning: This removes the volume attachment but does not delete the underlying data. The PersistentVolume and cloud storage remain for data recovery.

Forgetting a File Store​

Remove file store definition from Codiac:

cod filestore forget my-s3-bucket \
  --enterprise my-company

Result:

  • File store definition removed from Codiac
  • Does not delete cloud resources (S3 bucket, Azure storage account)
  • Use this when migrating storage or cleaning up unused definitions

Storage Classes​

Different storage tiers offer trade-offs between performance and cost:

Storage ClassPerformanceCostBest For
standardMediumLowGeneral-purpose, logs, backups
ssd / premiumHighHighDatabases, high-I/O workloads
cold / archiveLowVery LowLong-term backups, archives

Example: Choosing Storage Class

# High-performance database
cod asset volume create /data \
  --asset mongodb \
  --storage-class premium \
  --size 1Ti

# Log storage (lower cost)
cod asset volume create /var/log \
  --asset app-server \
  --storage-class standard \
  --size 50Gi

Best Practices​

1. Size Volumes Appropriately​

Oversizing:

  • ✅ Prevents out-of-space errors
  • ❌ Wastes money on unused storage

Recommendation:

  • Start with 2x current data size
  • Monitor usage and resize as needed

2. Use Appropriate Storage Classes​

SSD/Premium:

  • Databases (PostgreSQL, MongoDB, MySQL)
  • High-transaction systems
  • Real-time applications

Standard:

  • Application logs
  • File uploads
  • Temporary caches

3. Always Mount Databases with Volumes​

Without Volume:

Container Dies → All Data Lost ❌

With Volume:

Container Dies → Data Persists → New Container Reattaches → Data Intact ✅

4. Set Backup Policies​

Cloud storage includes snapshot and backup features:

  • Enable automated snapshots for critical data
  • Set retention policies (e.g., 30-day retention)
  • Test restore procedures regularly

Common File Store Patterns​

Pattern 1: PostgreSQL Database​

cod asset volume create /var/lib/postgresql/data \
  --asset postgres \
  --size 500Gi \
  --storage-class ssd

Why: Database files require high-performance SSD storage for IOPS.

Pattern 2: Redis Cache​

cod asset volume create /data \
  --asset redis \
  --size 50Gi \
  --storage-class ssd

Why: In-memory cache benefits from SSD for persistence (RDB/AOF files).

Pattern 3: File Upload API​

cod asset volume create /app/uploads \
  --asset api-server \
  --size 200Gi \
  --storage-class standard

Why: User uploads can use standard storage to save costs.

Pattern 4: Shared Assets (CMS)​

cod asset volume create /var/www/uploads \
  --asset cms \
  --size 100Gi \
  --storage-class standard \
  --access-mode ReadWriteMany

Why: Multiple CMS pods need shared access to uploaded files.

Cloud Storage Integration​

AWS S3 Integration​

Use Case: Object storage for media, backups, user uploads.

Setup:

  1. Create S3 bucket in AWS
  2. Capture bucket in Codiac: 
    cod filestore capture --provider aws --name my-bucket
  3. Use S3 SDK in application code to access bucket

Application Access:

const AWS = require('aws-sdk');
const s3 = new AWS.S3();

// Upload to S3 bucket captured in Codiac
s3.putObject({
  Bucket: 'my-bucket',
  Key: 'uploads/file.pdf',
  Body: fileBuffer
});

Azure Blob Storage Integration​

Use Case: Scalable object storage for files, backups.

Setup:

  1. Create Azure Storage Account
  2. Capture in Codiac: 
    cod filestore capture --provider azure --name my-storage-account
  3. Use Azure SDK in application

Application Access:

const { BlobServiceClient } = require('@azure/storage-blob');

const blobService = BlobServiceClient.fromConnectionString(process.env.AZURE_STORAGE);
const container = blobService.getContainerClient('uploads');
await container.uploadBlob('file.pdf', fileBuffer);

Troubleshooting​

Problem: Volume stuck in "Pending" status​

Possible Causes:

  1. Storage class not available in cluster
  2. Insufficient cloud quota
  3. Invalid size or configuration

Solutions:

# Check available storage classes
kubectl get storageclass

# Describe PVC for error details
kubectl describe pvc my-volume -n prod

# Common fix: Ensure storage class exists
kubectl get sc | grep ssd

Problem: "Out of space" errors​

Solutions:

  1. Resize volume (increase size): 
    cod asset volume resize /data --size 1Ti
  2. Clean up old data
  3. Add log rotation for application logs

Problem: Poor database performance​

Cause: Using standard storage class instead of ssd.

Solution: Migrate to SSD storage:

  1. Create new volume with SSD storage class
  2. Backup database
  3. Restore to new volume
  4. Delete old volume

FAQ​

Q: Can I attach the same volume to multiple assets?

A: Only if volume has ReadWriteMany access mode. Most volumes are ReadWriteOnce (single pod write access).

Q: What happens to data if I delete an asset?

A: Volume remains in cloud storage. Data is not deleted unless you explicitly delete the PersistentVolume.

Q: Can I resize a volume?

A: Yes, you can expand volumes (increase size). Shrinking is not supported and requires data migration.

Q: Do volumes work with autoscaling?

A: Yes for ReadWriteMany volumes. ReadWriteOnce volumes can only be used by one pod at a time (not suitable for horizontal scaling).

Q: How do I backup volume data?

A: Use cloud-native snapshots (EBS snapshots, Azure disk snapshots) or application-level backups (pg_dump for PostgreSQL, mongodump for MongoDB).

Q: What's the difference between volumes and file stores?

A: File stores are cloud storage definitions (S3 buckets, etc.). Volumes are mounted storage attached to containers (PVCs). File stores can be used for object storage, while volumes provide block storage.

Need help with storage configuration? Contact Support or check our cloud integration guides.