Showcase
Project: MerlinsModelsMicroservices Architecture
A behind-the-scenes look at the system designed to host any number of models with a corresponding API. The architecture is designed to be modular and scalable, allowing for easy addition of new models and services.
Infrastructure & Models
Running containerized services on a personal Debian server, secure external access, and currently hosting efficient language models.Infrastructure
Docker containers on Debian 12 XPS, Cloudflare Tunnels for secure access, and Nginx reverse proxy.Available Models
DeepSeek-Coder-Distill 1.5BA 1.5B-parameter model optimized for efficient CPU inference, fine-tuned for coding tasks.More models coming soon.Core Architecture
Modular microservices with clear responsibilities, built for scalability and maintainability.API Gateway
Routes requests, handles JWT authentication, and orchestrates service calls.Auth Service
Manages users, roles, and JWT token generation with RBAC enforcement.Inference Service
Executes model inferences with configurable parameters and metrics tracking.Technical Highlights
Features that demonstrate production-readiness, security, and performance.API Gateway Capabilities
• Request routing · JWT middleware · CRUD operations · Job queuing · Health checksSecurity & Auth
• Bearer tokens · RBAC · Token validation · Protected routesModel Interface
• Configurable params · Version support · Usage metrics · Extended timeoutsCurrent Capabilities
• Hosting DeepSeek-Coder-Distill 1.5B · REST APIs with OpenAPI · Next.js chat UI · Cloudflare deploymentChat Interface
Try out the model below. Ask questions, request creative content, or test its capabilities!DeepSeek 1.5B
This version of DeepSeek AI model is a real dummy
Hello! I'm a 1.5B parameter language model powered by DeepSeek. How can I help you today?