Back to Blog

Redis as Primary Database: When and How to Use Redis Beyond Caching

Abhishek18 min read
Redis as Primary Database: When and How to Use Redis Beyond Caching

Explore Redis as a primary database solution. Learn about Redis Stack, data persistence, high availability, and real-world use cases beyond traditional caching.

Redis has evolved far beyond its origins as a simple in-memory cache. With Redis Stack and advanced persistence options, Redis can serve as a powerful primary database for specific use cases. This comprehensive guide explores when and how to leverage Redis as your main data store.

Table of Contents

  1. Redis Evolution: From Cache to Database
  2. Redis Stack Capabilities
  3. Data Persistence Strategies
  4. High Availability and Clustering
  5. Data Modeling in Redis
  6. Performance Characteristics
  7. Use Cases for Redis as Primary DB
  8. Migration Strategies
  9. Limitations and Considerations

Redis Evolution: From Cache to Database

Traditional Redis vs Modern Redis

javascript
// Traditional Redis usage (caching)
const traditionalRedis = {
  purpose: "Cache frequently accessed data",
  dataTypes: ["string", "list", "set", "hash", "zset"],
  persistence: "optional",
  queryCapabilities: "key-based lookups",
  useCases: ["session storage", "cache layer", "pub/sub"]
};

// Modern Redis as Primary Database
const modernRedis = {
  purpose: "Full-featured database solution",
  dataTypes: ["all traditional types", "JSON", "time-series", "probabilistic"],
  persistence: "multiple strategies available",
  queryCapabilities: ["full-text search", "aggregations", "graph queries"],
  useCases: ["real-time analytics", "document store", "search engine", "time-series DB"]
};

Key Capabilities That Enable Primary Database Use

  1. Advanced Data Types: JSON, Time Series, Probabilistic structures
  2. Full-Text Search: RediSearch module for complex queries
  3. Robust Persistence: AOF and RDB with configurable durability
  4. High Availability: Redis Sentinel and Redis Cluster
  5. ACID Transactions: Multi-command transactions with optimistic locking
  6. Indexing: Secondary indexes for non-key-based queries

Redis Stack Capabilities

Redis Stack Modules Overview

bash
# Redis Stack includes these modules
REDIS_MODULES=(
    "RedisJSON"      # Native JSON data type
    "RediSearch"     # Full-text search and indexing
    "RedisTimeSeries" # Time-series data structures
    "RedisBloom"     # Probabilistic data structures
    "RedisGraph"     # Graph database capabilities
    "RedisGears"     # Programmable data processing
)

# Installation
docker run -d --name redis-stack \
    -p 6379:6379 \
    -p 8001:8001 \
    redis/redis-stack:latest

RedisJSON: Document Database Capabilities

javascript
// RedisJSON enables document storage and queries
const { createClient } = require('redis');

const client = createClient({
    url: 'redis://localhost:6379'
});

await client.connect();

// Store complex JSON documents
const userDocument = {
    id: "user:1001",
    name: "John Doe",
    email: "[email protected]",
    profile: {
        age: 30,
        location: "San Francisco",
        preferences: ["tech", "music", "travel"]
    },
    orders: [
        { id: "order:001", amount: 299.99, date: "2024-01-15" },
        { id: "order:002", amount: 149.50, date: "2024-02-20" }
    ],
    metadata: {
        lastLogin: "2024-03-01T10:30:00Z",
        accountType: "premium"
    }
};

// Store document
await client.json.set('user:1001', '$', userDocument);

// Query operations
const userName = await client.json.get('user:1001', { path: '$.name' });
const userAge = await client.json.get('user:1001', { path: '$.profile.age' });

// Update nested fields
await client.json.set('user:1001', '$.profile.age', 31);

// Array operations
await client.json.arrAppend('user:1001', '$.profile.preferences', 'photography');

// Complex queries with JSONPath
const recentOrders = await client.json.get('user:1001', { 
    path: '$.orders[?(@.date > "2024-02-01")]' 
});

console.log('Recent orders:', recentOrders);

RediSearch: Full-Text Search and Indexing

javascript
// Create search indexes for complex queries
async function setupSearchIndexes() {
    // Create index for user documents
    try {
        await client.ft.create('idx:users', {
            '$.name': {
                type: 'TEXT',
                SORTABLE: true
            },
            '$.email': {
                type: 'TEXT'
            },
            '$.profile.age': {
                type: 'NUMERIC',
                SORTABLE: true
            },
            '$.profile.location': {
                type: 'TEXT'
            },
            '$.metadata.accountType': {
                type: 'TAG'
            },
            '$.orders[*].amount': {
                type: 'NUMERIC'
            }
        }, {
            ON: 'JSON',
            PREFIX: 'user:'
        });
    } catch (e) {
        if (e.message === 'Index already exists') {
            console.log('Index already exists');
        } else {
            throw e;
        }
    }
}

// Perform complex searches
async function searchUsers() {
    // Text search
    const techUsers = await client.ft.search('idx:users', 'tech');
    
    // Numeric range queries
    const youngUsers = await client.ft.search('idx:users', '@profile.age:[18 25]');
    
    // Tag filtering
    const premiumUsers = await client.ft.search('idx:users', '@metadata.accountType:{premium}');
    
    // Complex combined queries
    const complexQuery = await client.ft.search('idx:users', 
        '@profile.location:"San Francisco" @profile.age:[25 35] @metadata.accountType:{premium}',
        {
            SORTBY: {
                BY: 'profile.age',
                DIRECTION: 'ASC'
            },
            LIMIT: {
                from: 0,
                size: 10
            }
        }
    );
    
    return complexQuery;
}

RedisTimeSeries: Time-Series Database

javascript
// Time-series data handling
async function setupTimeSeries() {
    // Create time-series for metrics
    await client.ts.create('metrics:cpu:server1', {
        RETENTION: 86400000, // 24 hours in milliseconds
        LABELS: {
            server: 'server1',
            metric: 'cpu',
            datacenter: 'us-east-1'
        }
    });
    
    await client.ts.create('metrics:memory:server1', {
        RETENTION: 86400000,
        LABELS: {
            server: 'server1', 
            metric: 'memory',
            datacenter: 'us-east-1'
        }
    });
}

// Add time-series data
async function addMetrics() {
    const now = Date.now();
    
    // Add CPU usage data
    await client.ts.add('metrics:cpu:server1', now, 45.2);
    await client.ts.add('metrics:cpu:server1', now + 60000, 52.1);
    await client.ts.add('metrics:cpu:server1', now + 120000, 38.7);
    
    // Add memory usage data
    await client.ts.add('metrics:memory:server1', now, 78.5);
    await client.ts.add('metrics:memory:server1', now + 60000, 82.1);
    await client.ts.add('metrics:memory:server1', now + 120000, 79.3);
}

// Query time-series data
async function queryMetrics() {
    // Get recent data
    const recentCPU = await client.ts.range('metrics:cpu:server1', 
        Date.now() - 3600000, // Last hour
        Date.now()
    );
    
    // Get aggregated data
    const avgCPU = await client.ts.range('metrics:cpu:server1',
        Date.now() - 86400000, // Last 24 hours
        Date.now(),
        {
            AGGREGATION: {
                type: 'avg',
                timeBucket: 3600000 // 1 hour buckets
            }
        }
    );
    
    // Multi-series queries
    const allMetrics = await client.ts.mRange(
        Date.now() - 3600000,
        Date.now(),
        'server=server1'
    );
    
    return { recentCPU, avgCPU, allMetrics };
}

Data Persistence Strategies

RDB (Redis Database) Snapshots

bash
# redis.conf configuration for RDB
save 900 1      # Save if at least 1 key changed in 900 seconds
save 300 10     # Save if at least 10 keys changed in 300 seconds  
save 60 10000   # Save if at least 10000 keys changed in 60 seconds

rdbcompression yes
rdbchecksum yes
dbfilename dump.rdb
dir /var/lib/redis/

# Manual snapshots
redis-cli BGSAVE      # Background save
redis-cli LASTSAVE    # Get timestamp of last save

AOF (Append Only File) Logging

bash
# redis.conf configuration for AOF
appendonly yes
appendfilename "appendonly.aof"

# Fsync policies
appendfsync always     # Fsync every write (slowest, safest)
appendfsync everysec   # Fsync every second (balanced)
appendfsync no         # Let OS decide when to fsync (fastest, least safe)

# AOF rewrite configuration
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb

# AOF rewrite commands
redis-cli BGREWRITEAOF  # Manually trigger AOF rewrite

Hybrid Persistence Strategy

bash
# Combined RDB + AOF for maximum durability
# redis.conf
save 900 1
save 300 10
save 60 10000

appendonly yes
appendfsync everysec
aof-use-rdb-preamble yes  # Use RDB format for AOF rewrites (faster)

# Backup strategy script
#!/bin/bash
BACKUP_DIR="/backups/redis/$(date +%Y%m%d)"
mkdir -p $BACKUP_DIR

# Create RDB snapshot
redis-cli BGSAVE
sleep 10

# Copy files
cp /var/lib/redis/dump.rdb $BACKUP_DIR/
cp /var/lib/redis/appendonly.aof $BACKUP_DIR/

# Compress and upload
tar -czf $BACKUP_DIR.tar.gz $BACKUP_DIR
aws s3 cp $BACKUP_DIR.tar.gz s3://redis-backups/

Data Durability Testing

javascript
// Test data durability across restarts
async function testDataDurability() {
    const testKey = 'durability:test';
    const testData = {
        timestamp: Date.now(),
        data: 'This should persist across restarts',
        counter: 12345
    };
    
    // Store test data
    await client.json.set(testKey, '$', testData);
    console.log('Data stored, restart Redis now...');
    
    // Simulate application restart
    await client.disconnect();
    
    // Reconnect and verify data persistence
    await client.connect();
    const retrievedData = await client.json.get(testKey);
    
    console.log('Data after restart:', retrievedData);
    console.log('Data persisted:', JSON.stringify(retrievedData) === JSON.stringify(testData));
}

High Availability and Clustering

Redis Sentinel Configuration

bash
# sentinel.conf
port 26379
sentinel monitor mymaster 127.0.0.1 6379 2
sentinel down-after-milliseconds mymaster 5000
sentinel failover-timeout mymaster 10000
sentinel parallel-syncs mymaster 1

# Authentication
sentinel auth-pass mymaster mypassword

# Start Sentinel
redis-server /path/to/sentinel.conf --sentinel
javascript
// Connect to Redis with Sentinel
const { createClient } = require('redis');

const client = createClient({
    socket: {
        host: 'localhost',
        port: 26379
    },
    name: 'mymaster',
    sentinelUsername: 'sentinel-user',
    sentinelPassword: 'sentinel-password',
    username: 'redis-user',
    password: 'redis-password'
});

// Handle failover events
client.on('error', (err) => {
    console.log('Redis Client Error', err);
});

client.on('connect', () => {
    console.log('Connected to Redis');
});

client.on('reconnecting', () => {
    console.log('Reconnecting to Redis...');
});

Redis Cluster Setup

bash
# Create Redis Cluster with 6 nodes (3 masters, 3 replicas)
# Start 6 Redis instances on different ports
for port in 7000 7001 7002 7003 7004 7005; do
    mkdir -p cluster/$port
    cat > cluster/$port/redis.conf << EOF
port $port
cluster-enabled yes
cluster-config-file nodes-$port.conf
cluster-node-timeout 5000
appendonly yes
appendfilename appendonly-$port.aof
dbfilename dump-$port.rdb
dir cluster/$port/
EOF
    redis-server cluster/$port/redis.conf &
done

# Create cluster
redis-cli --cluster create \
    127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 \
    127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 \
    --cluster-replicas 1
javascript
// Connect to Redis Cluster
const cluster = createCluster({
    rootNodes: [
        { url: 'redis://127.0.0.1:7000' },
        { url: 'redis://127.0.0.1:7001' },
        { url: 'redis://127.0.0.1:7002' }
    ],
    defaults: {
        password: 'cluster-password'
    }
});

await cluster.connect();

// Cluster operations automatically handle sharding
await cluster.json.set('user:1001', '$', userDocument);
const userData = await cluster.json.get('user:1001');

// Check cluster status
const clusterInfo = await cluster.cluster.info();
const clusterNodes = await cluster.cluster.nodes();
console.log('Cluster Info:', clusterInfo);
console.log('Cluster Nodes:', clusterNodes);

Data Modeling in Redis

Document-Based Data Modeling

javascript
// E-commerce data model using RedisJSON
const ecommerceModels = {
    // Product catalog
    product: {
        key: 'product:{productId}',
        structure: {
            id: 'string',
            name: 'string',
            description: 'string',
            price: 'number',
            categories: ['array'],
            attributes: {
                color: 'string',
                size: 'string',
                weight: 'number'
            },
            inventory: {
                stock: 'number',
                warehouse: 'string',
                reserved: 'number'
            },
            reviews: [{
                userId: 'string',
                rating: 'number',
                comment: 'string',
                date: 'string'
            }]
        }
    },
    
    // User profiles
    user: {
        key: 'user:{userId}',
        structure: {
            id: 'string',
            email: 'string',
            profile: {
                firstName: 'string',
                lastName: 'string',
                phone: 'string',
                addresses: [{
                    type: 'string',
                    street: 'string',
                    city: 'string',
                    country: 'string',
                    zipCode: 'string'
                }]
            },
            preferences: {
                categories: ['array'],
                priceRange: { min: 'number', max: 'number' },
                notifications: 'boolean'
            },
            orderHistory: ['array']
        }
    },
    
    // Shopping cart
    cart: {
        key: 'cart:{userId}',
        structure: {
            userId: 'string',
            items: [{
                productId: 'string',
                quantity: 'number',
                price: 'number',
                addedAt: 'string'
            }],
            totals: {
                subtotal: 'number',
                tax: 'number',
                shipping: 'number',
                total: 'number'
            },
            updatedAt: 'string'
        }
    }
};

// Implementation example
async function createEcommerceData() {
    // Create product
    const product = {
        id: 'prod:001',
        name: 'Wireless Headphones',
        description: 'High-quality wireless headphones with noise cancellation',
        price: 299.99,
        categories: ['Electronics', 'Audio', 'Headphones'],
        attributes: {
            color: 'Black',
            batteryLife: '30 hours',
            weight: 250
        },
        inventory: {
            stock: 50,
            warehouse: 'WH-001',
            reserved: 5
        },
        reviews: []
    };
    
    await client.json.set('product:001', '$', product);
    
    // Create user
    const user = {
        id: 'user:001',
        email: '[email protected]',
        profile: {
            firstName: 'John',
            lastName: 'Doe',
            phone: '+1-555-0123',
            addresses: [{
                type: 'home',
                street: '123 Main St',
                city: 'San Francisco',
                country: 'USA',
                zipCode: '94102'
            }]
        },
        preferences: {
            categories: ['Electronics', 'Books'],
            priceRange: { min: 0, max: 500 },
            notifications: true
        },
        orderHistory: []
    };
    
    await client.json.set('user:001', '$', user);
    
    // Create shopping cart
    const cart = {
        userId: 'user:001',
        items: [{
            productId: 'prod:001',
            quantity: 1,
            price: 299.99,
            addedAt: new Date().toISOString()
        }],
        totals: {
            subtotal: 299.99,
            tax: 24.00,
            shipping: 9.99,
            total: 333.98
        },
        updatedAt: new Date().toISOString()
    };
    
    await client.json.set('cart:user:001', '$', cart);
}

Search Index for E-commerce

javascript
// Create comprehensive search indexes
async function createEcommerceIndexes() {
    // Product search index
    await client.ft.create('idx:products', {
        '$.name': { type: 'TEXT', SORTABLE: true },
        '$.description': { type: 'TEXT' },
        '$.price': { type: 'NUMERIC', SORTABLE: true },
        '$.categories[*]': { type: 'TAG' },
        '$.attributes.color': { type: 'TAG' },
        '$.inventory.stock': { type: 'NUMERIC' }
    }, {
        ON: 'JSON',
        PREFIX: 'product:'
    });
    
    // User search index
    await client.ft.create('idx:users', {
        '$.email': { type: 'TEXT' },
        '$.profile.firstName': { type: 'TEXT' },
        '$.profile.lastName': { type: 'TEXT' },
        '$.profile.addresses[*].city': { type: 'TAG' },
        '$.preferences.categories[*]': { type: 'TAG' }
    }, {
        ON: 'JSON',
        PREFIX: 'user:'
    });
}

// E-commerce search operations
async function searchProducts(query) {
    // Text search with filters
    const searchResults = await client.ft.search('idx:products',
        `${query} @inventory.stock:[1 +inf]`, // Only in-stock items
        {
            SORTBY: {
                BY: 'price',
                DIRECTION: 'ASC'
            },
            LIMIT: { from: 0, size: 20 }
        }
    );
    
    return searchResults;
}

// Advanced filtering
async function filterProducts(filters) {
    let queryParts = [];
    
    if (filters.category) {
        queryParts.push(`@categories:{${filters.category}}`);
    }
    
    if (filters.priceMin && filters.priceMax) {
        queryParts.push(`@price:[${filters.priceMin} ${filters.priceMax}]`);
    }
    
    if (filters.color) {
        queryParts.push(`@attributes.color:{${filters.color}}`);
    }
    
    const query = queryParts.join(' ');
    
    return await client.ft.search('idx:products', query, {
        SORTBY: { BY: 'price', DIRECTION: 'ASC' }
    });
}

Performance Characteristics

Benchmarking Redis as Primary Database

javascript
// Performance testing suite
async function benchmarkRedisOperations() {
    const operations = 10000;
    const testData = {
        id: Math.random().toString(36),
        name: 'Test Document',
        data: Array(100).fill(0).map(() => Math.random()),
        timestamp: Date.now()
    };
    
    // JSON SET operations
    console.time('JSON SET Operations');
    for (let i = 0; i < operations; i++) {
        await client.json.set(`test:${i}`, '$', { ...testData, id: i });
    }
    console.timeEnd('JSON SET Operations');
    
    // JSON GET operations
    console.time('JSON GET Operations');
    for (let i = 0; i < operations; i++) {
        await client.json.get(`test:${i}`);
    }
    console.timeEnd('JSON GET Operations');
    
    // Search operations
    console.time('Search Operations');
    for (let i = 0; i < 100; i++) {
        await client.ft.search('idx:test', '*');
    }
    console.timeEnd('Search Operations');
    
    // Clean up
    await client.flushDb();
}

// Memory usage analysis
async function analyzeMemoryUsage() {
    const info = await client.info('memory');
    const memoryStats = {};
    
    info.split('\r\n').forEach(line => {
        if (line.includes(':')) {
            const [key, value] = line.split(':');
            if (key.startsWith('used_memory')) {
                memoryStats[key] = value;
            }
        }
    });
    
    return memoryStats;
}

Performance Optimization Techniques

bash
# redis.conf optimizations for primary database use
# Memory management
maxmemory 8gb
maxmemory-policy allkeys-lru  # Or noeviction for primary DB

# CPU optimization
hz 10  # Background task frequency
tcp-keepalive 300

# Network optimization
tcp-backlog 511
timeout 0

# Persistence optimization for primary DB
save 900 1
save 300 10
save 60 10000
appendonly yes
appendfsync everysec

# Disable dangerous commands in production
rename-command FLUSHDB ""
rename-command FLUSHALL ""
rename-command EVAL ""

Use Cases for Redis as Primary DB

Real-Time Analytics Platform

javascript
// Real-time analytics with RedisTimeSeries and RedisJSON
class RealTimeAnalytics {
    constructor(redisClient) {
        this.client = redisClient;
    }
    
    async trackEvent(userId, eventType, eventData) {
        const timestamp = Date.now();
        
        // Store event in time series
        await this.client.ts.add(
            `events:${eventType}:count`,
            timestamp,
            1
        );
        
        // Store detailed event data
        await this.client.json.set(
            `event:${timestamp}:${userId}`,
            '$',
            {
                userId,
                eventType,
                data: eventData,
                timestamp: new Date(timestamp).toISOString()
            }
        );
        
        // Update user activity
        await this.client.json.set(
            `user:${userId}:lastActivity`,
            '$',
            {
                eventType,
                timestamp: new Date(timestamp).toISOString()
            }
        );
    }
    
    async getAnalytics(eventType, timeRange) {
        // Get time series data
        const timeSeries = await this.client.ts.range(
            `events:${eventType}:count`,
            timeRange.start,
            timeRange.end,
            {
                AGGREGATION: {
                    type: 'sum',
                    timeBucket: 3600000 // 1 hour buckets
                }
            }
        );
        
        // Get top users for this event type
        const topUsers = await this.client.ft.search(
            'idx:events',
            `@eventType:{${eventType}} @timestamp:[${timeRange.start} ${timeRange.end}]`,
            {
                GROUPBY: {
                    properties: ['@userId'],
                    REDUCE: [{ function: 'COUNT', as: 'event_count' }]
                },
                SORTBY: { BY: '@event_count', DIRECTION: 'DESC' },
                LIMIT: { from: 0, size: 10 }
            }
        );
        
        return { timeSeries, topUsers };
    }
}

Session Store with Advanced Features

javascript
// Advanced session management
class RedisSessionStore {
    constructor(redisClient) {
        this.client = redisClient;
        this.ttl = 3600; // 1 hour default TTL
    }
    
    async createSession(userId, sessionData) {
        const sessionId = this.generateSessionId();
        const session = {
            id: sessionId,
            userId,
            data: sessionData,
            createdAt: new Date().toISOString(),
            lastAccessed: new Date().toISOString(),
            ipAddress: sessionData.ipAddress,
            userAgent: sessionData.userAgent
        };
        
        // Store session
        await this.client.json.set(`session:${sessionId}`, '$', session);
        await this.client.expire(`session:${sessionId}`, this.ttl);
        
        // Index by user ID for multi-session management
        await this.client.sAdd(`user:${userId}:sessions`, sessionId);
        
        return sessionId;
    }
    
    async getSession(sessionId) {
        const session = await this.client.json.get(`session:${sessionId}`);
        
        if (session) {
            // Update last accessed time
            await this.client.json.set(
                `session:${sessionId}`,
                '$.lastAccessed',
                new Date().toISOString()
            );
            
            // Refresh TTL
            await this.client.expire(`session:${sessionId}`, this.ttl);
        }
        
        return session;
    }
    
    async getUserSessions(userId) {
        const sessionIds = await this.client.sMembers(`user:${userId}:sessions`);
        const sessions = [];
        
        for (const sessionId of sessionIds) {
            const session = await this.client.json.get(`session:${sessionId}`);
            if (session) {
                sessions.push(session);
            } else {
                // Clean up expired session reference
                await this.client.sRem(`user:${userId}:sessions`, sessionId);
            }
        }
        
        return sessions;
    }
    
    generateSessionId() {
        return require('crypto').randomBytes(32).toString('hex');
    }
}

Content Management System

javascript
// CMS with full-text search capabilities
class RedisCMS {
    constructor(redisClient) {
        this.client = redisClient;
    }
    
    async createArticle(article) {
        const articleId = `article:${Date.now()}`;
        const articleData = {
            id: articleId,
            title: article.title,
            content: article.content,
            author: article.author,
            tags: article.tags,
            category: article.category,
            status: article.status || 'draft',
            createdAt: new Date().toISOString(),
            updatedAt: new Date().toISOString(),
            views: 0,
            likes: 0
        };
        
        await this.client.json.set(articleId, '$', articleData);
        
        // Add to category set
        await this.client.sAdd(`category:${article.category}`, articleId);
        
        // Add to author's articles
        await this.client.sAdd(`author:${article.author}:articles`, articleId);
        
        return articleId;
    }
    
    async searchArticles(query, filters = {}) {
        let searchQuery = query;
        
        if (filters.category) {
            searchQuery += ` @category:{${filters.category}}`;
        }
        
        if (filters.author) {
            searchQuery += ` @author:{${filters.author}}`;
        }
        
        if (filters.status) {
            searchQuery += ` @status:{${filters.status}}`;
        }
        
        const results = await this.client.ft.search('idx:articles', searchQuery, {
            HIGHLIGHT: {
                FIELDS: ['title', 'content'],
                TAGS: ['<mark>', '</mark>']
            },
            SORTBY: { BY: 'updatedAt', DIRECTION: 'DESC' },
            LIMIT: { from: 0, size: 20 }
        });
        
        return results;
    }
    
    async getPopularArticles(limit = 10) {
        return await this.client.ft.search('idx:articles', 
            '@status:{published}',
            {
                SORTBY: { BY: 'views', DIRECTION: 'DESC' },
                LIMIT: { from: 0, size: limit }
            }
        );
    }
}

Migration Strategies

From Traditional Database to Redis

javascript
// Migration from SQL database to Redis
class DatabaseMigration {
    constructor(sqlDb, redisClient) {
        this.sqlDb = sqlDb;
        this.redis = redisClient;
    }
    
    async migrateUsers() {
        const users = await this.sqlDb.query('SELECT * FROM users');
        
        for (const user of users) {
            const redisUser = {
                id: user.id,
                email: user.email,
                profile: {
                    firstName: user.first_name,
                    lastName: user.last_name,
                    phone: user.phone,
                    createdAt: user.created_at
                },
                preferences: JSON.parse(user.preferences || '{}'),
                isActive: user.is_active
            };
            
            await this.redis.json.set(`user:${user.id}`, '$', redisUser);
            
            // Create indexes during migration
            if (users.indexOf(user) === 0) {
                await this.createUserIndex();
            }
        }
        
        console.log(`Migrated ${users.length} users`);
    }
    
    async createUserIndex() {
        await this.redis.ft.create('idx:users', {
            '$.email': { type: 'TEXT' },
            '$.profile.firstName': { type: 'TEXT' },
            '$.profile.lastName': { type: 'TEXT' },
            '$.isActive': { type: 'TAG' }
        }, {
            ON: 'JSON',
            PREFIX: 'user:'
        });
    }
    
    async validateMigration() {
        // Compare counts
        const sqlCount = await this.sqlDb.query('SELECT COUNT(*) as count FROM users');
        const redisCount = await this.redis.ft.search('idx:users', '*', { LIMIT: { from: 0, size: 0 } });
        
        console.log(`SQL users: ${sqlCount[0].count}`);
        console.log(`Redis users: ${redisCount.total}`);
        
        return sqlCount[0].count === redisCount.total;
    }
}

Gradual Migration Strategy

javascript
// Dual-write pattern for gradual migration
class DualWriteService {
    constructor(primaryDb, redisClient) {
        this.primary = primaryDb;
        this.redis = redisClient;
        this.writeToRedis = process.env.WRITE_TO_REDIS === 'true';
        this.readFromRedis = process.env.READ_FROM_REDIS === 'true';
    }
    
    async createUser(userData) {
        let result;
        
        // Always write to primary database
        result = await this.primary.createUser(userData);
        
        // Conditionally write to Redis
        if (this.writeToRedis) {
            try {
                await this.redis.json.set(`user:${result.id}`, '$', {
                    id: result.id,
                    email: userData.email,
                    profile: {
                        firstName: userData.firstName,
                        lastName: userData.lastName
                    },
                    createdAt: new Date().toISOString()
                });
            } catch (error) {
                console.error('Redis write failed:', error);
                // Don't fail the operation if Redis write fails
            }
        }
        
        return result;
    }
    
    async getUser(userId) {
        if (this.readFromRedis) {
            try {
                const user = await this.redis.json.get(`user:${userId}`);
                if (user) {
                    return user;
                }
            } catch (error) {
                console.error('Redis read failed:', error);
            }
        }
        
        // Fallback to primary database
        return await this.primary.getUser(userId);
    }
}

Limitations and Considerations

When NOT to Use Redis as Primary Database

javascript
const limitations = {
    memoryConstraints: {
        issue: "All data must fit in memory",
        impact: "Cost increases linearly with data size",
        mitigations: ["Data archiving", "Tiered storage", "Selective caching"]
    },
    
    complexQueries: {
        issue: "Limited support for complex relational queries",
        impact: "May require application-level joins",
        alternatives: ["Denormalize data", "Use multiple queries", "Hybrid approach"]
    },
    
    transactions: {
        issue: "Limited ACID transaction support",
        impact: "Complex multi-document transactions not supported",
        workarounds: ["Lua scripts", "Optimistic locking", "Saga pattern"]
    },
    
    dataConsistency: {
        issue: "Eventual consistency in cluster mode",
        impact: "Read-after-write consistency not guaranteed",
        solutions: ["Read from master", "Application-level checks"]
    }
};

Cost Analysis for Large Datasets

javascript
// Cost comparison for different data sizes
function calculateRedisCosts(dataSizeGB, operationsPerSecond) {
    const costs = {
        cloudRedis: {
            // AWS ElastiCache pricing (approximate)
            memory: dataSizeGB * 50, // $50 per GB per month
            operations: operationsPerSecond * 0.0001 * 86400 * 30, // $0.0001 per 1000 ops
            total: function() { return this.memory + this.operations; }
        },
        
        selfHosted: {
            // Self-hosted costs
            memory: dataSizeGB * 20, // $20 per GB per month (hardware amortization)
            operations: 0, // No per-operation costs
            maintenance: 2000, // Monthly operational costs
            total: function() { return this.memory + this.operations + this.maintenance; }
        },
        
        traditional: {
            // Traditional database for comparison
            storage: dataSizeGB * 10, // $10 per GB per month
            compute: operationsPerSecond * 0.00005 * 86400 * 30,
            total: function() { return this.storage + this.compute; }
        }
    };
    
    return {
        redis_cloud: costs.cloudRedis.total(),
        redis_selfhosted: costs.selfHosted.total(),
        traditional_db: costs.traditional.total()
    };
}

// Example calculations
console.log('100GB dataset, 10K ops/sec:');
console.log(calculateRedisCosts(100, 10000));

console.log('1TB dataset, 50K ops/sec:');
console.log(calculateRedisCosts(1000, 50000));

Migration Risk Assessment

javascript
const migrationRisks = {
    dataLoss: {
        risk: "High",
        mitigation: [
            "Comprehensive backup strategy",
            "Dual-write during migration",
            "Rollback procedures"
        ]
    },
    
    performance: {
        risk: "Medium", 
        mitigation: [
            "Load testing before migration",
            "Gradual traffic shift",
            "Performance monitoring"
        ]
    },
    
    complexity: {
        risk: "Medium",
        mitigation: [
            "Team training on Redis",
            "Documentation updates",
            "Monitoring setup"
        ]
    },
    
    vendor_lockin: {
        risk: "Medium",
        mitigation: [
            "Use standard Redis APIs",
            "Avoid cloud-specific features",
            "Maintain export capabilities"
        ]
    }
};

Conclusion

Redis as a primary database is viable for specific use cases where:

Choose Redis as Primary Database when:

  • You need sub-millisecond response times
  • Your dataset fits comfortably in memory (with room for growth)
  • You require real-time analytics or time-series data
  • Your queries are primarily key-based or simple searches
  • You need built-in pub/sub capabilities
  • Operational simplicity is important

Avoid Redis as Primary Database when:

  • You have complex relational data requirements
  • Your dataset is too large for memory-based storage
  • You need strong ACID transaction guarantees
  • Cost optimization is critical for large datasets
  • You require complex analytical queries

Hybrid Approaches:

Consider using Redis alongside traditional databases:

  • Redis for hot data and caching
  • Traditional DB for cold data and complex queries
  • Event-driven synchronization between systems

Redis as a primary database represents a significant architectural decision that can provide exceptional performance and simplicity for the right use cases. Careful evaluation of your specific requirements, constraints, and long-term scalability needs is essential for success.

For expert guidance on Redis architecture and implementation strategies, our team provides comprehensive Redis consulting services and database architecture review.


Considering Redis as your primary database? Our certified database architects can help you evaluate the feasibility and design the optimal solution for your use case. Contact us for a free consultation.

Written by

Abhishek

KEEP LEARNING WITH THEDBADMIN

Turn this guide into a skill you can prove

Keep reading

Featured course · New batch

NextGen DBA + AI Engineer Bootcamp

Instructor-led, lab-first training for production DBAs: PostgreSQL, MongoDB, Docker, Kubernetes and AI.

  • Live labs on real execution plans
  • Backup, PITR & HA in production
  • pgvector, RAG & AI for DBAs

🛒 FROM THE SHOP

TiDB for the MySQL DBA From Fundamentals to Advanced Operations

PDF · 34 pages

₹99₹4,99798% off

View product