admin/aiworker
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Complete documentation for future sessions

Browse Source 
- CLAUDE.md for AI agents to understand the codebase
- GITEA-GUIDE.md centralizes all Gitea operations (API, Registry, Auth)
- DEVELOPMENT-WORKFLOW.md explains complete dev process
- ROADMAP.md, NEXT-SESSION.md for planning
- QUICK-REFERENCE.md, TROUBLESHOOTING.md for daily use
- 40+ detailed docs in /docs folder
- Backend as submodule from Gitea

Everything documented for autonomous operation.

Co-Authored-By: Claude Sonnet 4.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Hector Ros
2026-01-20 00:36:53 +01:00
commit db71705842
49 changed files with 19162 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

484
docs/02-backend/api-endpoints.md Normal file
View File

@@ -0,0 +1,484 @@
# API Endpoints
## Base URL
```
http://localhost:3000/api
```
## Authentication
Todos los endpoints (excepto `/health`) requieren autenticación JWT:
```
Authorization: Bearer <token>
```
---
## Projects
### GET /projects
Lista todos los proyectos.
**Response**:
```json
{
"projects": [
{
"id": "uuid",
"name": "My Project",
"description": "Project description",
"giteaRepoUrl": "http://gitea/owner/repo",
"k8sNamespace": "my-project",
"status": "active",
"createdAt": "2026-01-19T10:00:00Z"
}
]
}
```
### GET /projects/:id
Obtiene detalles de un proyecto.
### POST /projects
Crea un nuevo proyecto.
**Body**:
```json
{
"name": "My New Project",
"description": "Project description",
"dockerImage": "node:20-alpine",
"envVars": {
"NODE_ENV": "production"
},
"replicas": 2,
"cpuLimit": "1000m",
"memoryLimit": "1Gi"
}
```
**Response**:
```json
{
"project": {
"id": "uuid",
"name": "My New Project",
"giteaRepoUrl": "http://gitea/owner/my-new-project",
"k8sNamespace": "my-new-project-abc123"
}
}
```
### PATCH /projects/:id
Actualiza un proyecto.
### DELETE /projects/:id
Elimina un proyecto y todos sus recursos.
---
## Tasks
### GET /tasks
Lista tareas con filtros opcionales.
**Query params**:
- `projectId`: Filtrar por proyecto
- `state`: Filtrar por estado (`backlog`, `in_progress`, etc.)
- `assignedAgentId`: Filtrar por agente
- `limit`: Límite de resultados (default: 50)
- `offset`: Offset para paginación
**Response**:
```json
{
"tasks": [
{
"id": "uuid",
"projectId": "uuid",
"title": "Implement login",
"description": "Create authentication system",
"state": "in_progress",
"priority": "high",
"assignedAgentId": "agent-123",
"branchName": "task-abc-implement-login",
"prNumber": 42,
"prUrl": "http://gitea/owner/repo/pulls/42",
"previewUrl": "https://task-abc.preview.aiworker.dev",
"createdAt": "2026-01-19T10:00:00Z"
}
],
"total": 10,
"limit": 50,
"offset": 0
}
```
### GET /tasks/:id
Obtiene detalles completos de una tarea incluyendo preguntas.
**Response**:
```json
{
"task": {
"id": "uuid",
"title": "Implement login",
"state": "needs_input",
"questions": [
{
"id": "q-uuid",
"question": "Which auth library should I use?",
"context": "Need to choose between JWT or session-based",
"askedAt": "2026-01-19T11:00:00Z",
"status": "pending"
}
],
"project": {
"name": "My Project",
"giteaRepoUrl": "..."
}
}
}
```
### POST /tasks
Crea una nueva tarea.
**Body**:
```json
{
"projectId": "uuid",
"title": "Implement feature X",
"description": "Detailed description...",
"priority": "high"
}
```
### PATCH /tasks/:id
Actualiza una tarea.
**Body**:
```json
{
"state": "approved",
"notes": "Looks good!"
}
```
### POST /tasks/:id/respond
Responde a una pregunta del agente.
**Body**:
```json
{
"questionId": "q-uuid",
"response": "Use JWT with jsonwebtoken library"
}
```
**Response**:
```json
{
"success": true,
"question": {
"id": "q-uuid",
"status": "answered",
"respondedAt": "2026-01-19T11:05:00Z"
}
}
```
### POST /tasks/:id/approve
Aprueba una tarea en estado `ready_to_test`.
### POST /tasks/:id/reject
Rechaza una tarea y la regresa a `in_progress`.
**Body**:
```json
{
"reason": "Needs more tests"
}
```
---
## Task Groups (Merges)
### POST /task-groups
Crea un grupo de tareas para merge a staging/production.
**Body**:
```json
{
"projectId": "uuid",
"taskIds": ["task-1", "task-2", "task-3"],
"targetBranch": "staging",
"notes": "Sprint 1 features"
}
```
**Response**:
```json
{
"taskGroup": {
"id": "uuid",
"taskIds": ["task-1", "task-2", "task-3"],
"status": "pending",
"stagingBranch": "release/sprint-1"
}
}
```
### GET /task-groups/:id
Obtiene detalles de un task group.
### POST /task-groups/:id/deploy-staging
Despliega el task group a staging.
### POST /task-groups/:id/deploy-production
Despliega el task group a production.
---
## Agents
### GET /agents
Lista todos los agentes.
**Response**:
```json
{
"agents": [
{
"id": "agent-123",
"podName": "claude-agent-abc123",
"status": "busy",
"currentTaskId": "task-uuid",
"capabilities": ["javascript", "react", "node"],
"tasksCompleted": 42,
"lastHeartbeat": "2026-01-19T12:00:00Z"
}
]
}
```
### GET /agents/:id
Obtiene detalles de un agente incluyendo logs recientes.
### GET /agents/:id/logs
Obtiene logs del agente.
**Query params**:
- `limit`: Número de logs (default: 100)
- `level`: Filtrar por nivel (`debug`, `info`, `warn`, `error`)
---
## Deployments
### GET /deployments
Lista deployments con filtros.
**Query params**:
- `projectId`: Filtrar por proyecto
- `environment`: Filtrar por entorno
- `status`: Filtrar por estado
### GET /deployments/:id
Obtiene detalles de un deployment.
### POST /deployments/:id/rollback
Hace rollback de un deployment.
**Response**:
```json
{
"success": true,
"rollbackDeploymentId": "new-uuid"
}
```
---
## Health & Status
### GET /health
Health check del backend.
**Response**:
```json
{
"status": "ok",
"timestamp": "2026-01-19T12:00:00Z",
"services": {
"mysql": "connected",
"redis": "connected",
"gitea": "reachable",
"kubernetes": "connected"
},
"version": "1.0.0"
}
```
### GET /metrics
Métricas del sistema (Prometheus format).
---
## WebSocket Events
Conectar a: `ws://localhost:3000`
### Client → Server
```json
{
"event": "auth",
"data": {
"token": "jwt-token"
}
}
```
```json
{
"event": "subscribe",
"data": {
"projectId": "uuid"
}
}
```
### Server → Client
```json
{
"event": "task:created",
"data": {
"taskId": "uuid",
"projectId": "uuid",
"title": "New task"
}
}
```
```json
{
"event": "task:status_changed",
"data": {
"taskId": "uuid",
"oldState": "in_progress",
"newState": "ready_to_test",
"previewUrl": "https://..."
}
}
```
```json
{
"event": "task:needs_input",
"data": {
"taskId": "uuid",
"questionId": "q-uuid",
"question": "Which library?"
}
}
```
```json
{
"event": "agent:status",
"data": {
"agentId": "agent-123",
"status": "idle",
"lastTaskId": "task-uuid"
}
}
```
```json
{
"event": "deploy:started",
"data": {
"deploymentId": "uuid",
"environment": "staging"
}
}
```
```json
{
"event": "deploy:completed",
"data": {
"deploymentId": "uuid",
"environment": "staging",
"url": "https://staging-project.aiworker.dev"
}
}
```
---
## Error Responses
Todos los endpoints pueden retornar estos errores:
### 400 Bad Request
```json
{
"error": "Validation error",
"details": {
"field": "projectId",
"message": "Required"
}
}
```
### 401 Unauthorized
```json
{
"error": "Invalid or expired token"
}
```
### 404 Not Found
```json
{
"error": "Resource not found"
}
```
### 500 Internal Server Error
```json
{
"error": "Internal server error",
"requestId": "req-uuid"
}
```

462
docs/02-backend/database-schema.md Normal file
View File

@@ -0,0 +1,462 @@
# Database Schema con Drizzle ORM
## Schema Definitions
```typescript
// db/schema.ts
import { relations } from 'drizzle-orm'
import {
mysqlTable,
varchar,
text,
timestamp,
json,
int,
mysqlEnum,
boolean,
bigint,
index,
unique,
} from 'drizzle-orm/mysql-core'
// ============================================
// PROJECTS TABLE
// ============================================
export const projects = mysqlTable('projects', {
id: varchar('id', { length: 36 }).primaryKey(),
name: varchar('name', { length: 255 }).notNull(),
description: text('description'),
// Gitea
giteaRepoId: int('gitea_repo_id'),
giteaRepoUrl: varchar('gitea_repo_url', { length: 512 }),
giteaOwner: varchar('gitea_owner', { length: 100 }),
giteaRepoName: varchar('gitea_repo_name', { length: 100 }),
defaultBranch: varchar('default_branch', { length: 100 }).default('main'),
// K8s
k8sNamespace: varchar('k8s_namespace', { length: 63 }).notNull().unique(),
// Infrastructure
dockerImage: varchar('docker_image', { length: 512 }),
envVars: json('env_vars').$type<Record<string, string>>(),
replicas: int('replicas').default(1),
cpuLimit: varchar('cpu_limit', { length: 20 }).default('500m'),
memoryLimit: varchar('memory_limit', { length: 20 }).default('512Mi'),
// MCP
mcpTools: json('mcp_tools').$type<string[]>(),
mcpPermissions: json('mcp_permissions').$type<Record<string, any>>(),
// Status
status: mysqlEnum('status', ['active', 'paused', 'archived']).default('active'),
// Timestamps
createdAt: timestamp('created_at').defaultNow(),
updatedAt: timestamp('updated_at').defaultNow().onUpdateNow(),
}, (table) => ({
statusIdx: index('idx_status').on(table.status),
k8sNamespaceIdx: index('idx_k8s_namespace').on(table.k8sNamespace),
}))
// ============================================
// AGENTS TABLE
// ============================================
export const agents = mysqlTable('agents', {
id: varchar('id', { length: 36 }).primaryKey(),
// K8s
podName: varchar('pod_name', { length: 253 }).notNull().unique(),
k8sNamespace: varchar('k8s_namespace', { length: 63 }).default('agents'),
nodeName: varchar('node_name', { length: 253 }),
// Status
status: mysqlEnum('status', ['idle', 'busy', 'error', 'offline', 'initializing']).default('initializing'),
currentTaskId: varchar('current_task_id', { length: 36 }),
// Capabilities
capabilities: json('capabilities').$type<string[]>(),
maxConcurrentTasks: int('max_concurrent_tasks').default(1),
// Health
lastHeartbeat: timestamp('last_heartbeat'),
errorMessage: text('error_message'),
restartsCount: int('restarts_count').default(0),
// Metrics
tasksCompleted: int('tasks_completed').default(0),
totalRuntimeMinutes: int('total_runtime_minutes').default(0),
// Timestamps
createdAt: timestamp('created_at').defaultNow(),
updatedAt: timestamp('updated_at').defaultNow().onUpdateNow(),
}, (table) => ({
statusIdx: index('idx_status').on(table.status),
podNameIdx: index('idx_pod_name').on(table.podName),
lastHeartbeatIdx: index('idx_last_heartbeat').on(table.lastHeartbeat),
}))
// ============================================
// TASKS TABLE
// ============================================
export const tasks = mysqlTable('tasks', {
id: varchar('id', { length: 36 }).primaryKey(),
projectId: varchar('project_id', { length: 36 }).notNull().references(() => projects.id, { onDelete: 'cascade' }),
// Task info
title: varchar('title', { length: 255 }).notNull(),
description: text('description'),
priority: mysqlEnum('priority', ['low', 'medium', 'high', 'urgent']).default('medium'),
// State
state: mysqlEnum('state', [
'backlog',
'in_progress',
'needs_input',
'ready_to_test',
'approved',
'staging',
'production',
'cancelled'
]).default('backlog'),
// Assignment
assignedAgentId: varchar('assigned_agent_id', { length: 36 }).references(() => agents.id, { onDelete: 'set null' }),
assignedAt: timestamp('assigned_at'),
// Git
branchName: varchar('branch_name', { length: 255 }),
prNumber: int('pr_number'),
prUrl: varchar('pr_url', { length: 512 }),
// Preview
previewNamespace: varchar('preview_namespace', { length: 63 }),
previewUrl: varchar('preview_url', { length: 512 }),
previewDeployedAt: timestamp('preview_deployed_at'),
// Metadata
estimatedComplexity: mysqlEnum('estimated_complexity', ['trivial', 'simple', 'medium', 'complex']).default('medium'),
actualDurationMinutes: int('actual_duration_minutes'),
// Timestamps
createdAt: timestamp('created_at').defaultNow(),
updatedAt: timestamp('updated_at').defaultNow().onUpdateNow(),
startedAt: timestamp('started_at'),
completedAt: timestamp('completed_at'),
deployedStagingAt: timestamp('deployed_staging_at'),
deployedProductionAt: timestamp('deployed_production_at'),
}, (table) => ({
projectStateIdx: index('idx_project_state').on(table.projectId, table.state, table.createdAt),
stateIdx: index('idx_state').on(table.state),
assignedAgentIdx: index('idx_assigned_agent').on(table.assignedAgentId),
createdAtIdx: index('idx_created_at').on(table.createdAt),
}))
// ============================================
// TASK QUESTIONS TABLE
// ============================================
export const taskQuestions = mysqlTable('task_questions', {
id: varchar('id', { length: 36 }).primaryKey(),
taskId: varchar('task_id', { length: 36 }).notNull().references(() => tasks.id, { onDelete: 'cascade' }),
// Question
question: text('question').notNull(),
context: text('context'),
askedAt: timestamp('asked_at').defaultNow(),
// Response
response: text('response'),
respondedAt: timestamp('responded_at'),
respondedBy: varchar('responded_by', { length: 36 }),
// Status
status: mysqlEnum('status', ['pending', 'answered', 'skipped']).default('pending'),
}, (table) => ({
taskStatusIdx: index('idx_task_status').on(table.taskId, table.status),
statusIdx: index('idx_status').on(table.status),
}))
// ============================================
// TASK GROUPS TABLE
// ============================================
export const taskGroups = mysqlTable('task_groups', {
id: varchar('id', { length: 36 }).primaryKey(),
projectId: varchar('project_id', { length: 36 }).notNull().references(() => projects.id, { onDelete: 'cascade' }),
// Grouping
taskIds: json('task_ids').$type<string[]>().notNull(),
// Staging
stagingBranch: varchar('staging_branch', { length: 255 }),
stagingPrNumber: int('staging_pr_number'),
stagingPrUrl: varchar('staging_pr_url', { length: 512 }),
stagingDeployedAt: timestamp('staging_deployed_at'),
// Production
productionDeployedAt: timestamp('production_deployed_at'),
productionRollbackAvailable: boolean('production_rollback_available').default(true),
// Status
status: mysqlEnum('status', ['pending', 'staging', 'production', 'rolled_back']).default('pending'),
// Metadata
createdBy: varchar('created_by', { length: 36 }),
notes: text('notes'),
// Timestamps
createdAt: timestamp('created_at').defaultNow(),
updatedAt: timestamp('updated_at').defaultNow().onUpdateNow(),
}, (table) => ({
projectStatusIdx: index('idx_project_status').on(table.projectId, table.status),
statusIdx: index('idx_status').on(table.status),
}))
// ============================================
// DEPLOYMENTS TABLE
// ============================================
export const deployments = mysqlTable('deployments', {
id: varchar('id', { length: 36 }).primaryKey(),
projectId: varchar('project_id', { length: 36 }).notNull().references(() => projects.id, { onDelete: 'cascade' }),
taskGroupId: varchar('task_group_id', { length: 36 }).references(() => taskGroups.id, { onDelete: 'set null' }),
// Deployment info
environment: mysqlEnum('environment', ['preview', 'staging', 'production']).notNull(),
deploymentType: mysqlEnum('deployment_type', ['manual', 'automatic', 'rollback']).default('manual'),
// Git
branch: varchar('branch', { length: 255 }),
commitHash: varchar('commit_hash', { length: 40 }),
// K8s
k8sNamespace: varchar('k8s_namespace', { length: 63 }),
k8sDeploymentName: varchar('k8s_deployment_name', { length: 253 }),
imageTag: varchar('image_tag', { length: 255 }),
// Status
status: mysqlEnum('status', ['pending', 'in_progress', 'completed', 'failed', 'rolled_back']).default('pending'),
// Results
url: varchar('url', { length: 512 }),
errorMessage: text('error_message'),
logs: text('logs'),
// Timing
startedAt: timestamp('started_at'),
completedAt: timestamp('completed_at'),
durationSeconds: int('duration_seconds'),
// Metadata
triggeredBy: varchar('triggered_by', { length: 36 }),
// Timestamps
createdAt: timestamp('created_at').defaultNow(),
}, (table) => ({
projectEnvIdx: index('idx_project_env').on(table.projectId, table.environment),
statusIdx: index('idx_status').on(table.status),
createdAtIdx: index('idx_created_at').on(table.createdAt),
}))
// ============================================
// AGENT LOGS TABLE
// ============================================
export const agentLogs = mysqlTable('agent_logs', {
id: bigint('id', { mode: 'number' }).autoincrement().primaryKey(),
agentId: varchar('agent_id', { length: 36 }).notNull().references(() => agents.id, { onDelete: 'cascade' }),
taskId: varchar('task_id', { length: 36 }).references(() => tasks.id, { onDelete: 'set null' }),
// Log entry
level: mysqlEnum('level', ['debug', 'info', 'warn', 'error']).default('info'),
message: text('message').notNull(),
metadata: json('metadata').$type<Record<string, any>>(),
// Timestamp
createdAt: timestamp('created_at').defaultNow(),
}, (table) => ({
agentCreatedIdx: index('idx_agent_created').on(table.agentId, table.createdAt),
taskCreatedIdx: index('idx_task_created').on(table.taskId, table.createdAt),
levelIdx: index('idx_level').on(table.level),
}))
// ============================================
// RELATIONS
// ============================================
export const projectsRelations = relations(projects, ({ many }) => ({
tasks: many(tasks),
taskGroups: many(taskGroups),
deployments: many(deployments),
}))
export const tasksRelations = relations(tasks, ({ one, many }) => ({
project: one(projects, {
fields: [tasks.projectId],
references: [projects.id],
}),
assignedAgent: one(agents, {
fields: [tasks.assignedAgentId],
references: [agents.id],
}),
questions: many(taskQuestions),
}))
export const agentsRelations = relations(agents, ({ one, many }) => ({
currentTask: one(tasks, {
fields: [agents.currentTaskId],
references: [tasks.id],
}),
logs: many(agentLogs),
}))
export const taskQuestionsRelations = relations(taskQuestions, ({ one }) => ({
task: one(tasks, {
fields: [taskQuestions.taskId],
references: [tasks.id],
}),
}))
export const taskGroupsRelations = relations(taskGroups, ({ one, many }) => ({
project: one(projects, {
fields: [taskGroups.projectId],
references: [projects.id],
}),
deployments: many(deployments),
}))
export const deploymentsRelations = relations(deployments, ({ one }) => ({
project: one(projects, {
fields: [deployments.projectId],
references: [projects.id],
}),
taskGroup: one(taskGroups, {
fields: [deployments.taskGroupId],
references: [taskGroups.id],
}),
}))
export const agentLogsRelations = relations(agentLogs, ({ one }) => ({
agent: one(agents, {
fields: [agentLogs.agentId],
references: [agents.id],
}),
task: one(tasks, {
fields: [agentLogs.taskId],
references: [tasks.id],
}),
}))
```
## Drizzle Configuration
```typescript
// drizzle.config.ts
import type { Config } from 'drizzle-kit'
export default {
schema: './src/db/schema.ts',
out: './drizzle/migrations',
driver: 'mysql2',
dbCredentials: {
host: process.env.DB_HOST || 'localhost',
port: parseInt(process.env.DB_PORT || '3306'),
user: process.env.DB_USER || 'root',
password: process.env.DB_PASSWORD || '',
database: process.env.DB_NAME || 'aiworker',
},
} satisfies Config
```
## Database Client
```typescript
// db/client.ts
import { drizzle } from 'drizzle-orm/mysql2'
import mysql from 'mysql2/promise'
import * as schema from './schema'
const pool = mysql.createPool({
host: process.env.DB_HOST,
port: parseInt(process.env.DB_PORT || '3306'),
user: process.env.DB_USER,
password: process.env.DB_PASSWORD,
database: process.env.DB_NAME,
waitForConnections: true,
connectionLimit: 10,
queueLimit: 0,
})
export const db = drizzle(pool, { schema, mode: 'default' })
```
## Ejemplos de Queries
```typescript
// Get all tasks for a project
const projectTasks = await db.query.tasks.findMany({
where: eq(tasks.projectId, projectId),
with: {
assignedAgent: true,
questions: {
where: eq(taskQuestions.status, 'pending')
}
},
orderBy: [desc(tasks.createdAt)]
})
// Get next available task
const nextTask = await db.query.tasks.findFirst({
where: eq(tasks.state, 'backlog'),
orderBy: [desc(tasks.priority), asc(tasks.createdAt)]
})
// Get idle agents
const idleAgents = await db.query.agents.findMany({
where: and(
eq(agents.status, 'idle'),
gt(agents.lastHeartbeat, new Date(Date.now() - 60000))
)
})
// Insert new task
const newTask = await db.insert(tasks).values({
id: crypto.randomUUID(),
projectId: projectId,
title: 'New task',
description: 'Task description',
state: 'backlog',
priority: 'medium',
})
```
## Migrations
```bash
# Generate migration
bun run drizzle-kit generate:mysql
# Push changes directly (dev only)
bun run drizzle-kit push:mysql
# Run migrations
bun run scripts/migrate.ts
```
```typescript
// scripts/migrate.ts
import { migrate } from 'drizzle-orm/mysql2/migrator'
import { db } from '../src/db/client'
async function runMigrations() {
await migrate(db, { migrationsFolder: './drizzle/migrations' })
console.log('✓ Migrations completed')
process.exit(0)
}
runMigrations().catch(console.error)
```

480
docs/02-backend/estructura.md Normal file
View File

@@ -0,0 +1,480 @@
# Estructura del Backend
## Árbol de Directorios
```
backend/
├── src/
│ ├── index.ts # Entry point
│ ├── config/
│ │ ├── database.ts # MySQL connection
│ │ ├── redis.ts # Redis connection
│ │ └── env.ts # Environment variables
│ │
│ ├── api/
│ │ ├── app.ts # Express app setup
│ │ ├── routes/
│ │ │ ├── index.ts
│ │ │ ├── projects.ts # /api/projects
│ │ │ ├── tasks.ts # /api/tasks
│ │ │ ├── agents.ts # /api/agents
│ │ │ ├── deployments.ts# /api/deployments
│ │ │ └── health.ts # /api/health
│ │ │
│ │ ├── middleware/
│ │ │ ├── auth.ts # JWT validation
│ │ │ ├── error.ts # Error handler
│ │ │ ├── logger.ts # Request logging
│ │ │ └── validate.ts # Schema validation
│ │ │
│ │ └── websocket/
│ │ ├── server.ts # Socket.io setup
│ │ └── handlers.ts # WS event handlers
│ │
│ ├── db/
│ │ ├── schema.ts # Drizzle schema
│ │ ├── migrations/ # SQL migrations
│ │ └── client.ts # DB client instance
│ │
│ ├── services/
│ │ ├── mcp/
│ │ │ ├── server.ts # MCP server for agents
│ │ │ ├── tools.ts # MCP tool definitions
│ │ │ └── handlers.ts # Tool implementations
│ │ │
│ │ ├── gitea/
│ │ │ ├── client.ts # Gitea API client
│ │ │ ├── repos.ts # Repo operations
│ │ │ ├── pulls.ts # PR operations
│ │ │ └── webhooks.ts # Webhook handling
│ │ │
│ │ ├── kubernetes/
│ │ │ ├── client.ts # K8s API client
│ │ │ ├── namespaces.ts # Namespace management
│ │ │ ├── deployments.ts# Deployment management
│ │ │ ├── pods.ts # Pod operations
│ │ │ └── ingress.ts # Ingress management
│ │ │
│ │ ├── queue/
│ │ │ ├── task-queue.ts # Task queue
│ │ │ ├── deploy-queue.ts# Deploy queue
│ │ │ └── workers.ts # Queue workers
│ │ │
│ │ └── cache/
│ │ ├── redis.ts # Redis operations
│ │ └── strategies.ts # Caching strategies
│ │
│ ├── models/
│ │ ├── Project.ts # Project model
│ │ ├── Task.ts # Task model
│ │ ├── Agent.ts # Agent model
│ │ ├── TaskGroup.ts # TaskGroup model
│ │ └── Deployment.ts # Deployment model
│ │
│ ├── types/
│ │ ├── api.ts # API types
│ │ ├── mcp.ts # MCP types
│ │ ├── k8s.ts # K8s types
│ │ └── common.ts # Common types
│ │
│ └── utils/
│ ├── logger.ts # Winston logger
│ ├── errors.ts # Custom errors
│ ├── validators.ts # Validation helpers
│ └── helpers.ts # General helpers
│
├── drizzle/ # Drizzle config
│ ├── drizzle.config.ts
│ └── migrations/
│
├── tests/
│ ├── unit/
│ ├── integration/
│ └── e2e/
│
├── scripts/
│ ├── seed.ts # Seed database
│ ├── migrate.ts # Run migrations
│ └── generate-types.ts # Generate types
│
├── .env.example
├── .eslintrc.json
├── .prettierrc
├── tsconfig.json
├── package.json
└── README.md
```
## Entry Point (index.ts)
```typescript
import { startServer } from './api/app'
import { connectDatabase } from './config/database'
import { connectRedis } from './config/redis'
import { startMCPServer } from './services/mcp/server'
import { startQueueWorkers } from './services/queue/workers'
import { logger } from './utils/logger'
async function bootstrap() {
try {
// Connect to MySQL
await connectDatabase()
logger.info('✓ MySQL connected')
// Connect to Redis
await connectRedis()
logger.info('✓ Redis connected')
// Start MCP Server for agents
await startMCPServer()
logger.info('✓ MCP Server started')
// Start BullMQ workers
await startQueueWorkers()
logger.info('✓ Queue workers started')
// Start HTTP + WebSocket server
await startServer()
logger.info('✓ API Server started on port 3000')
} catch (error) {
logger.error('Failed to start server:', error)
process.exit(1)
}
}
bootstrap()
```
## Express App Setup (api/app.ts)
```typescript
import express from 'express'
import cors from 'cors'
import { createServer } from 'http'
import { Server as SocketIOServer } from 'socket.io'
import routes from './routes'
import { errorHandler } from './middleware/error'
import { requestLogger } from './middleware/logger'
import { setupWebSocket } from './websocket/server'
export async function startServer() {
const app = express()
const httpServer = createServer(app)
const io = new SocketIOServer(httpServer, {
cors: { origin: process.env.FRONTEND_URL }
})
// Middleware
app.use(cors())
app.use(express.json())
app.use(requestLogger)
// Routes
app.use('/api', routes)
// Error handling
app.use(errorHandler)
// WebSocket
setupWebSocket(io)
// Start
const port = process.env.PORT || 3000
httpServer.listen(port)
return { app, httpServer, io }
}
```
## Configuración de Base de Datos
```typescript
// config/database.ts
import { drizzle } from 'drizzle-orm/mysql2'
import mysql from 'mysql2/promise'
import * as schema from '../db/schema'
let connection: mysql.Connection
let db: ReturnType<typeof drizzle>
export async function connectDatabase() {
connection = await mysql.createConnection({
host: process.env.DB_HOST,
port: parseInt(process.env.DB_PORT || '3306'),
user: process.env.DB_USER,
password: process.env.DB_PASSWORD,
database: process.env.DB_NAME,
})
db = drizzle(connection, { schema, mode: 'default' })
return db
}
export function getDatabase() {
if (!db) {
throw new Error('Database not initialized')
}
return db
}
```
## Configuración de Redis
```typescript
// config/redis.ts
import Redis from 'ioredis'
let redis: Redis
export async function connectRedis() {
redis = new Redis({
host: process.env.REDIS_HOST || 'localhost',
port: parseInt(process.env.REDIS_PORT || '6379'),
password: process.env.REDIS_PASSWORD,
retryStrategy: (times) => {
const delay = Math.min(times * 50, 2000)
return delay
}
})
await redis.ping()
return redis
}
export function getRedis() {
if (!redis) {
throw new Error('Redis not initialized')
}
return redis
}
```
## Variables de Entorno
```bash
# .env.example
# Server
NODE_ENV=development
PORT=3000
FRONTEND_URL=http://localhost:5173
# Database
DB_HOST=localhost
DB_PORT=3306
DB_USER=root
DB_PASSWORD=password
DB_NAME=aiworker
# Redis
REDIS_HOST=localhost
REDIS_PORT=6379
REDIS_PASSWORD=
# Gitea
GITEA_URL=http://localhost:3001
GITEA_TOKEN=your-gitea-token
GITEA_OWNER=aiworker
# Kubernetes
K8S_IN_CLUSTER=false
K8S_CONFIG_PATH=~/.kube/config
K8S_DEFAULT_NAMESPACE=aiworker
# MCP Server
MCP_SERVER_PORT=3100
MCP_AUTH_TOKEN=your-mcp-token
# JWT
JWT_SECRET=your-secret-key
JWT_EXPIRES_IN=7d
# Claude API
ANTHROPIC_API_KEY=your-api-key
```
## Scripts de Package.json
```json
{
"name": "aiworker-backend",
"version": "1.0.0",
"scripts": {
"dev": "bun --watch src/index.ts",
"build": "bun build src/index.ts --outdir dist --target node",
"start": "bun dist/index.js",
"db:generate": "drizzle-kit generate:mysql",
"db:push": "drizzle-kit push:mysql",
"db:migrate": "bun run scripts/migrate.ts",
"db:seed": "bun run scripts/seed.ts",
"test": "bun test",
"test:watch": "bun test --watch",
"lint": "eslint src/**/*.ts",
"format": "prettier --write src/**/*.ts"
},
"dependencies": {
"express": "^4.19.0",
"mysql2": "^3.11.0",
"drizzle-orm": "^0.36.0",
"ioredis": "^5.4.1",
"bullmq": "^5.23.0",
"socket.io": "^4.8.1",
"@modelcontextprotocol/sdk": "^1.0.0",
"@kubernetes/client-node": "^0.22.0",
"axios": "^1.7.9",
"zod": "^3.24.1",
"winston": "^3.17.0",
"jsonwebtoken": "^9.0.2",
"cors": "^2.8.5",
"dotenv": "^16.4.7"
},
"devDependencies": {
"@types/express": "^5.0.0",
"@types/node": "^22.10.2",
"drizzle-kit": "^0.31.0",
"typescript": "^5.7.2",
"prettier": "^3.4.2",
"eslint": "^9.18.0"
}
}
```
## Estructura de Rutas
```typescript
// api/routes/index.ts
import { Router } from 'express'
import projectRoutes from './projects'
import taskRoutes from './tasks'
import agentRoutes from './agents'
import deploymentRoutes from './deployments'
import healthRoutes from './health'
const router = Router()
router.use('/projects', projectRoutes)
router.use('/tasks', taskRoutes)
router.use('/agents', agentRoutes)
router.use('/deployments', deploymentRoutes)
router.use('/health', healthRoutes)
export default router
```
## Middleware de Validación
```typescript
// middleware/validate.ts
import { Request, Response, NextFunction } from 'express'
import { ZodSchema } from 'zod'
export function validate(schema: ZodSchema) {
return (req: Request, res: Response, next: NextFunction) => {
try {
schema.parse({
body: req.body,
query: req.query,
params: req.params,
})
next()
} catch (error) {
res.status(400).json({
error: 'Validation error',
details: error
})
}
}
}
```
## Logger Setup
```typescript
// utils/logger.ts
import winston from 'winston'
export const logger = winston.createLogger({
level: process.env.LOG_LEVEL || 'info',
format: winston.format.combine(
winston.format.timestamp(),
winston.format.errors({ stack: true }),
winston.format.json()
),
transports: [
new winston.transports.Console({
format: winston.format.combine(
winston.format.colorize(),
winston.format.simple()
)
}),
new winston.transports.File({ filename: 'error.log', level: 'error' }),
new winston.transports.File({ filename: 'combined.log' })
]
})
```
## Manejo de Errores
```typescript
// middleware/error.ts
import { Request, Response, NextFunction } from 'express'
import { logger } from '../utils/logger'
export class AppError extends Error {
statusCode: number
isOperational: boolean
constructor(message: string, statusCode: number) {
super(message)
this.statusCode = statusCode
this.isOperational = true
Error.captureStackTrace(this, this.constructor)
}
}
export function errorHandler(
err: Error | AppError,
req: Request,
res: Response,
next: NextFunction
) {
logger.error('Error:', err)
if (err instanceof AppError) {
return res.status(err.statusCode).json({
error: err.message
})
}
res.status(500).json({
error: 'Internal server error'
})
}
```
## Comandos Útiles
```bash
# Desarrollo
bun run dev
# Generar migraciones
bun run db:generate
# Aplicar migraciones
bun run db:migrate
# Seed inicial
bun run db:seed
# Tests
bun test
# Build para producción
bun run build
# Producción
bun run start
```

459
docs/02-backend/gitea-integration.md Normal file
View File

@@ -0,0 +1,459 @@
# Integración con Gitea
## Cliente de Gitea
```typescript
// services/gitea/client.ts
import axios, { AxiosInstance } from 'axios'
import { logger } from '../../utils/logger'
export interface GiteaConfig {
url: string
token: string
owner: string
}
export class GiteaClient {
private client: AxiosInstance
private owner: string
constructor(config?: GiteaConfig) {
const url = config?.url || process.env.GITEA_URL!
const token = config?.token || process.env.GITEA_TOKEN!
this.owner = config?.owner || process.env.GITEA_OWNER!
this.client = axios.create({
baseURL: `${url}/api/v1`,
headers: {
'Authorization': `token ${token}`,
'Content-Type': 'application/json'
},
timeout: 30000
})
// Log requests
this.client.interceptors.request.use((config) => {
logger.debug(`Gitea API: ${config.method?.toUpperCase()} ${config.url}`)
return config
})
// Handle errors
this.client.interceptors.response.use(
(response) => response,
(error) => {
logger.error('Gitea API Error:', {
url: error.config?.url,
status: error.response?.status,
data: error.response?.data
})
throw error
}
)
}
// ============================================
// REPOSITORIES
// ============================================
async createRepo(name: string, options: {
description?: string
private?: boolean
autoInit?: boolean
defaultBranch?: string
} = {}) {
const response = await this.client.post('/user/repos', {
name,
description: options.description || '',
private: options.private !== false,
auto_init: options.autoInit !== false,
default_branch: options.defaultBranch || 'main',
trust_model: 'default'
})
logger.info(`Gitea: Created repo ${name}`)
return response.data
}
async getRepo(owner: string, repo: string) {
const response = await this.client.get(`/repos/${owner}/${repo}`)
return response.data
}
async deleteRepo(owner: string, repo: string) {
await this.client.delete(`/repos/${owner}/${repo}`)
logger.info(`Gitea: Deleted repo ${owner}/${repo}`)
}
async listRepos(owner?: string) {
const targetOwner = owner || this.owner
const response = await this.client.get(`/users/${targetOwner}/repos`)
return response.data
}
// ============================================
// BRANCHES
// ============================================
async createBranch(owner: string, repo: string, branchName: string, fromBranch: string = 'main') {
// Get reference commit
const refResponse = await this.client.get(
`/repos/${owner}/${repo}/git/refs/heads/${fromBranch}`
)
const sha = refResponse.data.object.sha
// Create new branch
const response = await this.client.post(
`/repos/${owner}/${repo}/git/refs`,
{
ref: `refs/heads/${branchName}`,
sha
}
)
logger.info(`Gitea: Created branch ${branchName} from ${fromBranch}`)
return response.data
}
async getBranch(owner: string, repo: string, branch: string) {
const response = await this.client.get(
`/repos/${owner}/${repo}/branches/${branch}`
)
return response.data
}
async listBranches(owner: string, repo: string) {
const response = await this.client.get(
`/repos/${owner}/${repo}/branches`
)
return response.data
}
async deleteBranch(owner: string, repo: string, branch: string) {
await this.client.delete(
`/repos/${owner}/${repo}/branches/${branch}`
)
logger.info(`Gitea: Deleted branch ${branch}`)
}
// ============================================
// PULL REQUESTS
// ============================================
async createPullRequest(owner: string, repo: string, data: {
title: string
body: string
head: string
base: string
}) {
const response = await this.client.post(
`/repos/${owner}/${repo}/pulls`,
{
title: data.title,
body: data.body,
head: data.head,
base: data.base
}
)
logger.info(`Gitea: Created PR #${response.data.number}`)
return response.data
}
async getPullRequest(owner: string, repo: string, index: number) {
const response = await this.client.get(
`/repos/${owner}/${repo}/pulls/${index}`
)
return response.data
}
async listPullRequests(owner: string, repo: string, state: 'open' | 'closed' | 'all' = 'open') {
const response = await this.client.get(
`/repos/${owner}/${repo}/pulls`,
{ params: { state } }
)
return response.data
}
async mergePullRequest(owner: string, repo: string, index: number, method: 'merge' | 'rebase' | 'squash' = 'merge') {
const response = await this.client.post(
`/repos/${owner}/${repo}/pulls/${index}/merge`,
{
Do: method,
MergeMessageField: '',
MergeTitleField: ''
}
)
logger.info(`Gitea: Merged PR #${index}`)
return response.data
}
async closePullRequest(owner: string, repo: string, index: number) {
const response = await this.client.patch(
`/repos/${owner}/${repo}/pulls/${index}`,
{ state: 'closed' }
)
logger.info(`Gitea: Closed PR #${index}`)
return response.data
}
// ============================================
// COMMITS
// ============================================
async getCommit(owner: string, repo: string, sha: string) {
const response = await this.client.get(
`/repos/${owner}/${repo}/git/commits/${sha}`
)
return response.data
}
async listCommits(owner: string, repo: string, options: {
sha?: string
path?: string
page?: number
limit?: number
} = {}) {
const response = await this.client.get(
`/repos/${owner}/${repo}/commits`,
{ params: options }
)
return response.data
}
// ============================================
// WEBHOOKS
// ============================================
async createWebhook(owner: string, repo: string, config: {
url: string
contentType?: 'json' | 'form'
secret?: string
events?: string[]
}) {
const response = await this.client.post(
`/repos/${owner}/${repo}/hooks`,
{
type: 'gitea',
config: {
url: config.url,
content_type: config.contentType || 'json',
secret: config.secret || ''
},
events: config.events || ['push', 'pull_request'],
active: true
}
)
logger.info(`Gitea: Created webhook for ${owner}/${repo}`)
return response.data
}
async listWebhooks(owner: string, repo: string) {
const response = await this.client.get(
`/repos/${owner}/${repo}/hooks`
)
return response.data
}
async deleteWebhook(owner: string, repo: string, hookId: number) {
await this.client.delete(
`/repos/${owner}/${repo}/hooks/${hookId}`
)
logger.info(`Gitea: Deleted webhook ${hookId}`)
}
// ============================================
// FILES
// ============================================
async getFileContents(owner: string, repo: string, filepath: string, ref: string = 'main') {
const response = await this.client.get(
`/repos/${owner}/${repo}/contents/${filepath}`,
{ params: { ref } }
)
return response.data
}
async createOrUpdateFile(owner: string, repo: string, filepath: string, data: {
content: string // base64 encoded
message: string
branch?: string
sha?: string // for updates
}) {
const response = await this.client.post(
`/repos/${owner}/${repo}/contents/${filepath}`,
{
content: data.content,
message: data.message,
branch: data.branch || 'main',
sha: data.sha
}
)
logger.info(`Gitea: Updated file ${filepath}`)
return response.data
}
// ============================================
// USERS
// ============================================
async getCurrentUser() {
const response = await this.client.get('/user')
return response.data
}
async getUser(username: string) {
const response = await this.client.get(`/users/${username}`)
return response.data
}
// ============================================
// ORGANIZATIONS (if needed)
// ============================================
async createOrg(name: string, options: {
fullName?: string
description?: string
} = {}) {
const response = await this.client.post('/orgs', {
username: name,
full_name: options.fullName || name,
description: options.description || ''
})
logger.info(`Gitea: Created org ${name}`)
return response.data
}
}
// Export singleton instance
export const giteaClient = new GiteaClient()
```
## Webhook Handler
```typescript
// services/gitea/webhooks.ts
import { Request, Response } from 'express'
import crypto from 'crypto'
import { logger } from '../../utils/logger'
import { db } from '../../db/client'
import { tasks } from '../../db/schema'
import { eq } from 'drizzle-orm'
import { emitWebSocketEvent } from '../../api/websocket/server'
export async function handleGiteaWebhook(req: Request, res: Response) {
const signature = req.headers['x-gitea-signature'] as string
const event = req.headers['x-gitea-event'] as string
const payload = req.body
// Verify signature
const secret = process.env.GITEA_WEBHOOK_SECRET || ''
if (secret && signature) {
const hmac = crypto.createHmac('sha256', secret)
hmac.update(JSON.stringify(payload))
const calculatedSignature = hmac.digest('hex')
if (signature !== calculatedSignature) {
logger.warn('Gitea webhook: Invalid signature')
return res.status(401).json({ error: 'Invalid signature' })
}
}
logger.info(`Gitea webhook: ${event}`, {
repo: payload.repository?.full_name,
ref: payload.ref
})
try {
switch (event) {
case 'push':
await handlePushEvent(payload)
break
case 'pull_request':
await handlePullRequestEvent(payload)
break
default:
logger.debug(`Unhandled webhook event: ${event}`)
}
res.status(200).json({ success: true })
} catch (error) {
logger.error('Webhook handler error:', error)
res.status(500).json({ error: 'Internal error' })
}
}
async function handlePushEvent(payload: any) {
const branch = payload.ref.replace('refs/heads/', '')
const commits = payload.commits || []
logger.info(`Push to ${branch}: ${commits.length} commits`)
// Find task by branch name
const task = await db.query.tasks.findFirst({
where: eq(tasks.branchName, branch)
})
if (task) {
emitWebSocketEvent('task:push', {
taskId: task.id,
branch,
commitsCount: commits.length
})
}
}
async function handlePullRequestEvent(payload: any) {
const action = payload.action // opened, closed, reopened, edited, synchronized
const prNumber = payload.pull_request.number
const state = payload.pull_request.state
logger.info(`PR #${prNumber}: ${action}`)
// Find task by PR number
const task = await db.query.tasks.findFirst({
where: eq(tasks.prNumber, prNumber)
})
if (task) {
if (action === 'closed' && payload.pull_request.merged) {
// PR was merged
await db.update(tasks)
.set({ state: 'staging' })
.where(eq(tasks.id, task.id))
emitWebSocketEvent('task:merged', {
taskId: task.id,
prNumber
})
}
emitWebSocketEvent('task:pr_updated', {
taskId: task.id,
prNumber,
action,
state
})
}
}
```
## Router para Webhooks
```typescript
// api/routes/webhooks.ts
import { Router } from 'express'
import { handleGiteaWebhook } from '../../services/gitea/webhooks'
const router = Router()
router.post('/gitea', handleGiteaWebhook)
export default router
```

788
docs/02-backend/mcp-server.md Normal file
View File

@@ -0,0 +1,788 @@
# MCP Server para Agentes
El MCP (Model Context Protocol) Server es la interfaz que permite a los agentes Claude Code comunicarse con el backend y ejecutar operaciones.
## Arquitectura MCP
```
┌─────────────────┐ ┌─────────────────┐
│ Claude Code │ MCP Protocol │ MCP Server │
│ (Agent Pod) │◄──────────────────►│ (Backend) │
└─────────────────┘ └─────────────────┘
│
┌─────────────────────┼─────────────────────┐
│ │ │
┌────▼────┐ ┌────▼────┐ ┌────▼────┐
│ MySQL │ │ Gitea │ │ K8s │
└─────────┘ └─────────┘ └─────────┘
```
## Setup del MCP Server
```typescript
// services/mcp/server.ts
import { Server } from '@modelcontextprotocol/sdk/server/index.js'
import { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js'
import { CallToolRequestSchema, ListToolsRequestSchema } from '@modelcontextprotocol/sdk/types.js'
import { tools } from './tools'
import { handleToolCall } from './handlers'
import { logger } from '../../utils/logger'
export class AgentMCPServer {
private server: Server
constructor() {
this.server = new Server(
{
name: 'aiworker-orchestrator',
version: '1.0.0',
},
{
capabilities: {
tools: {},
},
}
)
this.setupHandlers()
}
private setupHandlers() {
// List available tools
this.server.setRequestHandler(ListToolsRequestSchema, async () => {
return {
tools: tools.map(tool => ({
name: tool.name,
description: tool.description,
inputSchema: tool.inputSchema,
}))
}
})
// Handle tool calls
this.server.setRequestHandler(CallToolRequestSchema, async (request) => {
const { name, arguments: args } = request.params
logger.info(`MCP: Tool called: ${name}`, { args })
try {
const result = await handleToolCall(name, args)
return result
} catch (error) {
logger.error(`MCP: Tool error: ${name}`, error)
return {
content: [{
type: 'text',
text: `Error: ${error.message}`
}],
isError: true
}
}
})
}
async start() {
const transport = new StdioServerTransport()
await this.server.connect(transport)
logger.info('MCP Server started')
}
}
// Start MCP server
let mcpServer: AgentMCPServer
export async function startMCPServer() {
mcpServer = new AgentMCPServer()
await mcpServer.start()
return mcpServer
}
export function getMCPServer() {
return mcpServer
}
```
## Definición de Herramientas
```typescript
// services/mcp/tools.ts
import { z } from 'zod'
export const tools = [
{
name: 'get_next_task',
description: 'Obtiene la siguiente tarea disponible de la cola',
inputSchema: {
type: 'object',
properties: {
agentId: {
type: 'string',
description: 'ID del agente solicitante'
},
capabilities: {
type: 'array',
items: { type: 'string' },
description: 'Capacidades del agente (ej: ["javascript", "react"])'
}
},
required: ['agentId']
}
},
{
name: 'update_task_status',
description: 'Actualiza el estado de una tarea',
inputSchema: {
type: 'object',
properties: {
taskId: {
type: 'string',
description: 'ID de la tarea'
},
status: {
type: 'string',
enum: ['in_progress', 'needs_input', 'ready_to_test', 'completed'],
description: 'Nuevo estado'
},
metadata: {
type: 'object',
description: 'Metadata adicional (duración, errores, etc.)'
}
},
required: ['taskId', 'status']
}
},
{
name: 'ask_user_question',
description: 'Solicita información al usuario',
inputSchema: {
type: 'object',
properties: {
taskId: {
type: 'string',
description: 'ID de la tarea'
},
question: {
type: 'string',
description: 'Pregunta para el usuario'
},
context: {
type: 'string',
description: 'Contexto adicional'
}
},
required: ['taskId', 'question']
}
},
{
name: 'check_question_response',
description: 'Verifica si el usuario ha respondido una pregunta',
inputSchema: {
type: 'object',
properties: {
taskId: {
type: 'string',
description: 'ID de la tarea'
}
},
required: ['taskId']
}
},
{
name: 'create_branch',
description: 'Crea una nueva rama en Gitea',
inputSchema: {
type: 'object',
properties: {
taskId: {
type: 'string',
description: 'ID de la tarea'
},
branchName: {
type: 'string',
description: 'Nombre de la rama (opcional, se genera automático)'
}
},
required: ['taskId']
}
},
{
name: 'create_pull_request',
description: 'Crea un Pull Request en Gitea',
inputSchema: {
type: 'object',
properties: {
taskId: {
type: 'string',
description: 'ID de la tarea'
},
title: {
type: 'string',
description: 'Título del PR'
},
description: {
type: 'string',
description: 'Descripción del PR'
}
},
required: ['taskId', 'title', 'description']
}
},
{
name: 'trigger_preview_deploy',
description: 'Despliega un preview environment en K8s',
inputSchema: {
type: 'object',
properties: {
taskId: {
type: 'string',
description: 'ID de la tarea'
}
},
required: ['taskId']
}
},
{
name: 'get_task_details',
description: 'Obtiene detalles completos de una tarea',
inputSchema: {
type: 'object',
properties: {
taskId: {
type: 'string',
description: 'ID de la tarea'
}
},
required: ['taskId']
}
},
{
name: 'log_activity',
description: 'Registra actividad del agente',
inputSchema: {
type: 'object',
properties: {
agentId: {
type: 'string',
description: 'ID del agente'
},
level: {
type: 'string',
enum: ['debug', 'info', 'warn', 'error'],
description: 'Nivel de log'
},
message: {
type: 'string',
description: 'Mensaje'
},
metadata: {
type: 'object',
description: 'Metadata adicional'
}
},
required: ['agentId', 'message']
}
},
{
name: 'heartbeat',
description: 'Envía heartbeat para indicar que el agente está activo',
inputSchema: {
type: 'object',
properties: {
agentId: {
type: 'string',
description: 'ID del agente'
},
status: {
type: 'string',
enum: ['idle', 'busy', 'error'],
description: 'Estado actual'
}
},
required: ['agentId', 'status']
}
}
]
```
## Implementación de Handlers
```typescript
// services/mcp/handlers.ts
import { db } from '../../db/client'
import { tasks, agents, taskQuestions, agentLogs } from '../../db/schema'
import { eq, and, desc, asc } from 'drizzle-orm'
import { GiteaClient } from '../gitea/client'
import { K8sClient } from '../kubernetes/client'
import { getRedis } from '../../config/redis'
import { emitWebSocketEvent } from '../../api/websocket/server'
import crypto from 'crypto'
const giteaClient = new GiteaClient()
const k8sClient = new K8sClient()
const redis = getRedis()
export async function handleToolCall(name: string, args: any) {
switch (name) {
case 'get_next_task':
return await getNextTask(args)
case 'update_task_status':
return await updateTaskStatus(args)
case 'ask_user_question':
return await askUserQuestion(args)
case 'check_question_response':
return await checkQuestionResponse(args)
case 'create_branch':
return await createBranch(args)
case 'create_pull_request':
return await createPullRequest(args)
case 'trigger_preview_deploy':
return await triggerPreviewDeploy(args)
case 'get_task_details':
return await getTaskDetails(args)
case 'log_activity':
return await logActivity(args)
case 'heartbeat':
return await heartbeat(args)
default:
throw new Error(`Unknown tool: ${name}`)
}
}
// ============================================
// TOOL IMPLEMENTATIONS
// ============================================
async function getNextTask(args: { agentId: string; capabilities?: string[] }) {
const { agentId } = args
// Get next task from backlog
const task = await db.query.tasks.findFirst({
where: eq(tasks.state, 'backlog'),
with: {
project: true
},
orderBy: [desc(tasks.priority), asc(tasks.createdAt)]
})
if (!task) {
return {
content: [{
type: 'text',
text: JSON.stringify({ message: 'No tasks available' })
}]
}
}
// Assign task to agent
await db.update(tasks)
.set({
state: 'in_progress',
assignedAgentId: agentId,
assignedAt: new Date(),
startedAt: new Date()
})
.where(eq(tasks.id, task.id))
await db.update(agents)
.set({
status: 'busy',
currentTaskId: task.id
})
.where(eq(agents.id, agentId))
// Emit WebSocket event
emitWebSocketEvent('task:status_changed', {
taskId: task.id,
oldState: 'backlog',
newState: 'in_progress',
agentId
})
// Cache invalidation
await redis.del(`task:${task.id}`)
await redis.del(`task:list:${task.projectId}`)
return {
content: [{
type: 'text',
text: JSON.stringify({
task: {
id: task.id,
title: task.title,
description: task.description,
priority: task.priority,
project: task.project
}
})
}]
}
}
async function updateTaskStatus(args: { taskId: string; status: string; metadata?: any }) {
const { taskId, status, metadata } = args
const updates: any = { state: status }
if (status === 'completed') {
updates.completedAt = new Date()
}
if (metadata?.durationMinutes) {
updates.actualDurationMinutes = metadata.durationMinutes
}
await db.update(tasks)
.set(updates)
.where(eq(tasks.id, taskId))
// If task completed, free up agent
if (status === 'completed' || status === 'ready_to_test') {
const task = await db.query.tasks.findFirst({
where: eq(tasks.id, taskId)
})
if (task?.assignedAgentId) {
await db.update(agents)
.set({
status: 'idle',
currentTaskId: null,
tasksCompleted: db.$sql`tasks_completed + 1`
})
.where(eq(agents.id, task.assignedAgentId))
}
}
emitWebSocketEvent('task:status_changed', {
taskId,
newState: status,
metadata
})
await redis.del(`task:${taskId}`)
return {
content: [{
type: 'text',
text: JSON.stringify({ success: true })
}]
}
}
async function askUserQuestion(args: { taskId: string; question: string; context?: string }) {
const { taskId, question, context } = args
// Update task state
await db.update(tasks)
.set({ state: 'needs_input' })
.where(eq(tasks.id, taskId))
// Insert question
const questionId = crypto.randomUUID()
await db.insert(taskQuestions).values({
id: questionId,
taskId,
question,
context,
status: 'pending'
})
// Notify frontend
emitWebSocketEvent('task:needs_input', {
taskId,
questionId,
question,
context
})
await redis.del(`task:${taskId}`)
return {
content: [{
type: 'text',
text: JSON.stringify({
success: true,
message: 'Question sent to user',
questionId
})
}]
}
}
async function checkQuestionResponse(args: { taskId: string }) {
const { taskId } = args
const question = await db.query.taskQuestions.findFirst({
where: and(
eq(taskQuestions.taskId, taskId),
eq(taskQuestions.status, 'answered')
),
orderBy: [desc(taskQuestions.respondedAt)]
})
if (!question || !question.response) {
return {
content: [{
type: 'text',
text: JSON.stringify({
hasResponse: false,
message: 'No response yet'
})
}]
}
}
// Update task back to in_progress
await db.update(tasks)
.set({ state: 'in_progress' })
.where(eq(tasks.id, taskId))
return {
content: [{
type: 'text',
text: JSON.stringify({
hasResponse: true,
response: question.response,
question: question.question
})
}]
}
}
async function createBranch(args: { taskId: string; branchName?: string }) {
const { taskId, branchName } = args
const task = await db.query.tasks.findFirst({
where: eq(tasks.id, taskId),
with: { project: true }
})
if (!task) {
throw new Error('Task not found')
}
const branch = branchName || `task-${taskId.slice(0, 8)}-${task.title.toLowerCase().replace(/\s+/g, '-').slice(0, 30)}`
// Create branch in Gitea
await giteaClient.createBranch(
task.project.giteaOwner!,
task.project.giteaRepoName!,
branch,
task.project.defaultBranch!
)
// Update task
await db.update(tasks)
.set({ branchName: branch })
.where(eq(tasks.id, taskId))
return {
content: [{
type: 'text',
text: JSON.stringify({
success: true,
branchName: branch,
repoUrl: task.project.giteaRepoUrl
})
}]
}
}
async function createPullRequest(args: { taskId: string; title: string; description: string }) {
const { taskId, title, description } = args
const task = await db.query.tasks.findFirst({
where: eq(tasks.id, taskId),
with: { project: true }
})
if (!task || !task.branchName) {
throw new Error('Task not found or branch not created')
}
const pr = await giteaClient.createPullRequest(
task.project.giteaOwner!,
task.project.giteaRepoName!,
{
title,
body: description,
head: task.branchName,
base: task.project.defaultBranch!
}
)
await db.update(tasks)
.set({
prNumber: pr.number,
prUrl: pr.html_url
})
.where(eq(tasks.id, taskId))
emitWebSocketEvent('task:pr_created', {
taskId,
prUrl: pr.html_url,
prNumber: pr.number
})
return {
content: [{
type: 'text',
text: JSON.stringify({
success: true,
prUrl: pr.html_url,
prNumber: pr.number
})
}]
}
}
async function triggerPreviewDeploy(args: { taskId: string }) {
const { taskId } = args
const task = await db.query.tasks.findFirst({
where: eq(tasks.id, taskId),
with: { project: true }
})
if (!task) {
throw new Error('Task not found')
}
const previewNamespace = `preview-task-${taskId.slice(0, 8)}`
const previewUrl = `https://${previewNamespace}.preview.aiworker.dev`
// Deploy to K8s
await k8sClient.createPreviewDeployment({
namespace: previewNamespace,
taskId,
projectId: task.projectId,
image: task.project.dockerImage!,
branch: task.branchName!,
envVars: task.project.envVars as Record<string, string>
})
await db.update(tasks)
.set({
state: 'ready_to_test',
previewNamespace,
previewUrl,
previewDeployedAt: new Date()
})
.where(eq(tasks.id, taskId))
emitWebSocketEvent('task:ready_to_test', {
taskId,
previewUrl
})
return {
content: [{
type: 'text',
text: JSON.stringify({
success: true,
previewUrl,
namespace: previewNamespace
})
}]
}
}
async function getTaskDetails(args: { taskId: string }) {
const { taskId } = args
const task = await db.query.tasks.findFirst({
where: eq(tasks.id, taskId),
with: {
project: true,
questions: true
}
})
if (!task) {
throw new Error('Task not found')
}
return {
content: [{
type: 'text',
text: JSON.stringify({ task })
}]
}
}
async function logActivity(args: { agentId: string; level?: string; message: string; metadata?: any }) {
const { agentId, level = 'info', message, metadata } = args
await db.insert(agentLogs).values({
agentId,
level: level as any,
message,
metadata
})
return {
content: [{
type: 'text',
text: JSON.stringify({ success: true })
}]
}
}
async function heartbeat(args: { agentId: string; status: string }) {
const { agentId, status } = args
await db.update(agents)
.set({
lastHeartbeat: new Date(),
status: status as any
})
.where(eq(agents.id, agentId))
return {
content: [{
type: 'text',
text: JSON.stringify({ success: true })
}]
}
}
```
## Uso desde Claude Code Agent
Desde el pod del agente, Claude Code usaría las herramientas así:
```bash
# En el pod del agente, configurar MCP
# claude-code config add-mcp-server aiworker stdio \
# "bun run /app/mcp-client.js"
# Ejemplo de uso en conversación con Claude Code:
# User: "Toma la siguiente tarea y trabaja en ella"
# Claude Code internamente llama:
# - get_next_task({ agentId: "agent-xyz" })
# - Si necesita info: ask_user_question({ taskId: "...", question: "..." })
# - Trabaja en el código
# - create_branch({ taskId: "..." })
# - (commits and pushes)
# - create_pull_request({ taskId: "...", title: "...", description: "..." })
# - trigger_preview_deploy({ taskId: "..." })
# - update_task_status({ taskId: "...", status: "ready_to_test" })
```

520
docs/02-backend/queue-system.md Normal file
View File

@@ -0,0 +1,520 @@
# Sistema de Colas con BullMQ
## Setup de BullMQ
```typescript
// services/queue/config.ts
import { Queue, Worker, QueueScheduler } from 'bullmq'
import { getRedis } from '../../config/redis'
import { logger } from '../../utils/logger'
const connection = getRedis()
export const queues = {
tasks: new Queue('tasks', { connection }),
deploys: new Queue('deploys', { connection }),
merges: new Queue('merges', { connection }),
cleanup: new Queue('cleanup', { connection }),
}
// Queue options
export const defaultJobOptions = {
attempts: 3,
backoff: {
type: 'exponential',
delay: 2000,
},
removeOnComplete: {
age: 3600, // 1 hour
count: 1000,
},
removeOnFail: {
age: 86400, // 24 hours
},
}
```
## Task Queue
```typescript
// services/queue/task-queue.ts
import { queues, defaultJobOptions } from './config'
import { logger } from '../../utils/logger'
export interface TaskJob {
taskId: string
projectId: string
priority: 'low' | 'medium' | 'high' | 'urgent'
}
export async function enqueueTask(data: TaskJob) {
const priorityMap = {
urgent: 1,
high: 2,
medium: 3,
low: 4,
}
await queues.tasks.add('process-task', data, {
...defaultJobOptions,
priority: priorityMap[data.priority],
jobId: data.taskId,
})
logger.info(`Task queued: ${data.taskId}`)
}
export async function dequeueTask(taskId: string) {
const job = await queues.tasks.getJob(taskId)
if (job) {
await job.remove()
logger.info(`Task dequeued: ${taskId}`)
}
}
export async function getQueuedTasks() {
const jobs = await queues.tasks.getJobs(['waiting', 'active'])
return jobs.map(job => ({
id: job.id,
data: job.data,
state: await job.getState(),
progress: job.progress,
attemptsMade: job.attemptsMade,
}))
}
```
## Deploy Queue
```typescript
// services/queue/deploy-queue.ts
import { queues, defaultJobOptions } from './config'
import { logger } from '../../utils/logger'
export interface DeployJob {
deploymentId: string
projectId: string
taskId?: string
environment: 'preview' | 'staging' | 'production'
branch: string
commitHash: string
}
export async function enqueueDeploy(data: DeployJob) {
await queues.deploys.add('deploy', data, {
...defaultJobOptions,
priority: data.environment === 'production' ? 1 : 2,
jobId: data.deploymentId,
})
logger.info(`Deploy queued: ${data.environment} - ${data.deploymentId}`)
}
export async function getDeployStatus(deploymentId: string) {
const job = await queues.deploys.getJob(deploymentId)
if (!job) return null
return {
id: job.id,
state: await job.getState(),
progress: job.progress,
result: job.returnvalue,
failedReason: job.failedReason,
}
}
```
## Merge Queue
```typescript
// services/queue/merge-queue.ts
import { queues, defaultJobOptions } from './config'
import { logger } from '../../utils/logger'
export interface MergeJob {
taskGroupId: string
projectId: string
taskIds: string[]
targetBranch: 'staging' | 'main'
}
export async function enqueueMerge(data: MergeJob) {
await queues.merges.add('merge-tasks', data, {
...defaultJobOptions,
priority: data.targetBranch === 'main' ? 1 : 2,
jobId: data.taskGroupId,
})
logger.info(`Merge queued: ${data.taskGroupId}`)
}
```
## Cleanup Queue
```typescript
// services/queue/cleanup-queue.ts
import { queues, defaultJobOptions } from './config'
import { logger } from '../../utils/logger'
export interface CleanupJob {
type: 'preview-namespace' | 'old-logs' | 'completed-jobs'
namespaceOrResource: string
ageHours: number
}
export async function enqueueCleanup(data: CleanupJob) {
await queues.cleanup.add('cleanup', data, {
...defaultJobOptions,
attempts: 1,
})
logger.info(`Cleanup queued: ${data.type}`)
}
// Schedule recurring cleanup
export async function scheduleRecurringCleanup() {
// Clean preview namespaces older than 7 days
await queues.cleanup.add(
'cleanup-preview-namespaces',
{
type: 'preview-namespace',
ageHours: 168, // 7 days
},
{
repeat: {
pattern: '0 2 * * *', // Daily at 2 AM
},
}
)
// Clean old logs
await queues.cleanup.add(
'cleanup-old-logs',
{
type: 'old-logs',
ageHours: 720, // 30 days
},
{
repeat: {
pattern: '0 3 * * 0', // Weekly on Sunday at 3 AM
},
}
)
logger.info('Recurring cleanup jobs scheduled')
}
```
## Workers Implementation
```typescript
// services/queue/workers.ts
import { Worker, Job } from 'bullmq'
import { getRedis } from '../../config/redis'
import { logger } from '../../utils/logger'
import { db } from '../../db/client'
import { tasks, agents, deployments } from '../../db/schema'
import { eq } from 'drizzle-orm'
import { K8sClient } from '../kubernetes/client'
import { GiteaClient } from '../gitea/client'
import { TaskJob, DeployJob, MergeJob, CleanupJob } from './types'
const connection = getRedis()
const k8sClient = new K8sClient()
const giteaClient = new GiteaClient()
// ============================================
// TASK WORKER
// ============================================
const taskWorker = new Worker(
'tasks',
async (job: Job<TaskJob>) => {
logger.info(`Processing task job: ${job.id}`)
// Check if there's an available agent
const availableAgent = await db.query.agents.findFirst({
where: eq(agents.status, 'idle'),
})
if (!availableAgent) {
logger.info('No available agents, task will be retried')
throw new Error('No available agents')
}
// Task will be picked up by agent via MCP get_next_task
logger.info(`Task ${job.data.taskId} ready for agent pickup`)
return { success: true, readyForPickup: true }
},
{
connection,
concurrency: 5,
}
)
taskWorker.on('completed', (job) => {
logger.info(`Task job completed: ${job.id}`)
})
taskWorker.on('failed', (job, err) => {
logger.error(`Task job failed: ${job?.id}`, err)
})
// ============================================
// DEPLOY WORKER
// ============================================
const deployWorker = new Worker(
'deploys',
async (job: Job<DeployJob>) => {
const { deploymentId, projectId, environment, branch, commitHash } = job.data
logger.info(`Deploying: ${environment} - ${deploymentId}`)
// Update deployment status
await db.update(deployments)
.set({
status: 'in_progress',
startedAt: new Date(),
})
.where(eq(deployments.id, deploymentId))
job.updateProgress(10)
try {
// Get project config
const project = await db.query.projects.findFirst({
where: eq(deployments.projectId, projectId),
})
if (!project) {
throw new Error('Project not found')
}
job.updateProgress(20)
// Prepare deployment
const namespace = environment === 'production'
? `${project.k8sNamespace}-prod`
: environment === 'staging'
? `${project.k8sNamespace}-staging`
: job.data.taskId
? `preview-task-${job.data.taskId.slice(0, 8)}`
: project.k8sNamespace
job.updateProgress(40)
// Deploy to K8s
await k8sClient.createOrUpdateDeployment({
namespace,
name: `${project.name}-${environment}`,
image: `${project.dockerImage}:${commitHash.slice(0, 7)}`,
envVars: project.envVars as Record<string, string>,
replicas: project.replicas || 1,
resources: {
cpu: project.cpuLimit || '500m',
memory: project.memoryLimit || '512Mi',
},
})
job.updateProgress(70)
// Create/update ingress
const url = await k8sClient.createOrUpdateIngress({
namespace,
name: `${project.name}-${environment}`,
host: environment === 'production'
? `${project.name}.aiworker.dev`
: `${environment}-${project.name}.aiworker.dev`,
serviceName: `${project.name}-${environment}`,
servicePort: 3000,
})
job.updateProgress(90)
// Update deployment record
await db.update(deployments)
.set({
status: 'completed',
completedAt: new Date(),
url,
durationSeconds: Math.floor(
(new Date().getTime() - job.processedOn!) / 1000
),
})
.where(eq(deployments.id, deploymentId))
job.updateProgress(100)
logger.info(`Deploy completed: ${environment} - ${url}`)
return { success: true, url }
} catch (error) {
// Update deployment as failed
await db.update(deployments)
.set({
status: 'failed',
errorMessage: error.message,
completedAt: new Date(),
})
.where(eq(deployments.id, deploymentId))
throw error
}
},
{
connection,
concurrency: 3,
}
)
// ============================================
// MERGE WORKER
// ============================================
const mergeWorker = new Worker(
'merges',
async (job: Job<MergeJob>) => {
const { taskGroupId, projectId, taskIds, targetBranch } = job.data
logger.info(`Merging tasks: ${taskIds.join(', ')} to ${targetBranch}`)
// Get project and tasks
const project = await db.query.projects.findFirst({
where: eq(deployments.projectId, projectId),
})
if (!project) {
throw new Error('Project not found')
}
const tasksList = await db.query.tasks.findMany({
where: (tasks, { inArray }) => inArray(tasks.id, taskIds),
})
job.updateProgress(20)
// Merge each PR
for (const task of tasksList) {
if (task.prNumber) {
await giteaClient.mergePullRequest(
project.giteaOwner!,
project.giteaRepoName!,
task.prNumber,
'squash'
)
job.updateProgress(20 + (40 / tasksList.length))
}
}
job.updateProgress(60)
// Create staging/production branch if needed
// Then trigger deploy
// ... implementation
job.updateProgress(100)
logger.info(`Merge completed: ${taskGroupId}`)
return { success: true }
},
{
connection,
concurrency: 2,
}
)
// ============================================
// CLEANUP WORKER
// ============================================
const cleanupWorker = new Worker(
'cleanup',
async (job: Job<CleanupJob>) => {
const { type, ageHours } = job.data
logger.info(`Cleanup: ${type}`)
switch (type) {
case 'preview-namespace':
await k8sClient.cleanupOldPreviewNamespaces(ageHours)
break
case 'old-logs':
const cutoffDate = new Date(Date.now() - ageHours * 60 * 60 * 1000)
await db.delete(agentLogs)
.where(lt(agentLogs.createdAt, cutoffDate))
break
}
logger.info(`Cleanup completed: ${type}`)
return { success: true }
},
{
connection,
concurrency: 1,
}
)
// ============================================
// START ALL WORKERS
// ============================================
export async function startQueueWorkers() {
logger.info('Starting BullMQ workers...')
// Workers are already instantiated above
// Just schedule recurring jobs
await scheduleRecurringCleanup()
logger.info('✓ All workers started')
return {
taskWorker,
deployWorker,
mergeWorker,
cleanupWorker,
}
}
// Graceful shutdown
process.on('SIGTERM', async () => {
logger.info('Shutting down workers...')
await taskWorker.close()
await deployWorker.close()
await mergeWorker.close()
await cleanupWorker.close()
logger.info('Workers shut down')
process.exit(0)
})
```
## Monitorización de Colas
```typescript
// api/routes/queues.ts
import { Router } from 'express'
import { queues } from '../../services/queue/config'
const router = Router()
router.get('/status', async (req, res) => {
const status = await Promise.all(
Object.entries(queues).map(async ([name, queue]) => ({
name,
waiting: await queue.getWaitingCount(),
active: await queue.getActiveCount(),
completed: await queue.getCompletedCount(),
failed: await queue.getFailedCount(),
}))
)
res.json({ queues: status })
})
export default router
```