Local Development Environment Setup¶
This guide provides step-by-step instructions for setting up a complete SYNDI development environment from scratch. Essential for new developers, contributors, or when setting up isolated development environments.
Prerequisites¶
Before starting, ensure you have:
Conda (Miniconda or Anaconda) - Python environment management
Node.js (v18 or later) - Frontend development
Git - Version control
AWS CLI (optional) - For deployment and testing
jq (optional) - JSON processing for scripts
Quick Start¶
1. Clone the Repository¶
git clone <repository-url>
cd 2CLAIRE
2. Create the Syndi Environment¶
The project includes a pre-configured conda environment file matching AWS Lambda runtime (Python 3.9):
# Create syndi environment from configuration
conda env create -f environment.yml
# Activate the environment
conda activate syndi
3. Verify Installation¶
# Check Python version (should be 3.9.x)
python --version
# Verify key dependencies
python -c "import fastapi, pydantic, boto3; print('✅ Dependencies OK')"
4. Install Frontend Dependencies¶
# Install Node.js packages
cd frontend
npm install
# Install Playwright browsers for E2E testing
npx playwright install
cd ..
5. Setup Local Environment¶
# Create local directories and deploy configs
make setup-local ENV=dev ORG=myorg
This creates:
.local/s3/ # Simulated S3 buckets
├── forms/ # SOP forms and templates
├── eln/ # ELN submissions
├── eln-drafts/ # Draft submissions
├── public/ # Public assets
├── webapp/ # Built frontend (after build)
└── lambda/ # Packaged Lambda (after build)
6. Start Development Servers¶
# Start both backend and frontend with hot reload
make start-dev ENV=dev ORG=myorg
URLs:
Frontend:
http://localhost:3000(orhttp://localhost:5173)Backend:
http://localhost:8000API Docs:
http://localhost:8000/docs
Python Environments¶
syndi Environment (Python 3.9)¶
The syndi environment matches AWS Lambda runtime:
Python Version: 3.9.x (exactly matches AWS Lambda)
Key Dependencies: FastAPI, Pydantic, Boto3, Mangum
Purpose: Production simulation and Lambda deployment
Required for:
serve-lambdacommand
When to use syndi:
conda activate syndi
make serve-lambda ENV=dev ORG=myorg
claire Environment (Python 3.11)¶
Alternative environment for general development:
Python Version: 3.11+ (latest features)
Purpose: General development with newer Python features
Optional: Not required for SYNDI development
When to use claire:
conda activate claire
make start-backend ENV=dev ORG=myorg
Note: Per AGENTS.md, use syndi for serve-lambda (production simulation) and claire for development.
Switching Environments¶
# Switch to syndi (Python 3.9 - Lambda compatible)
conda activate syndi
# Switch to claire (Python 3.11 - General development)
conda activate claire
# Check current environment
conda env list
# Deactivate
conda deactivate
Environment Configuration¶
Environment Variables¶
The project uses environment-based configuration:
ENV: Environment identifierdev- Local developmenttest- Automated testingstage- Staging/pre-productionprod- Production
ORG: Organization identifierRequired for all commands (no default for security)
Examples:
myorg,uga,pwbUsed in resource naming and config selection
Configuration Files¶
After make setup-local, you’ll have:
backend/rawscribe/.config/config.json # Merged Lambda config
frontend/public/config.json # Merged webapp config
These are generated from:
infra/.config/lambda/dev.json # Base Lambda config
infra/.config/lambda/dev-myorg.json # Org-specific overrides
infra/.config/webapp/dev.json # Base webapp config
infra/.config/webapp/dev-myorg.json # Org-specific overrides
See Configuration System for details.
Updating Dependencies¶
Backend Dependencies¶
# Add package to requirements.txt
echo "pandas==2.0.0" >> backend/layers/dependencies/requirements.txt
# Reinstall in conda environment
conda activate syndi
pip install -r backend/layers/dependencies/requirements.txt
Frontend Dependencies¶
# Add package
cd frontend
npm install <package-name>
# Or edit package.json and run
npm install
Update Environment¶
If environment.yml changes:
# Update syndi environment
conda activate syndi
conda env update -f environment.yml --prune
# Or recreate from scratch
conda deactivate
conda env remove -n syndi
conda env create -f environment.yml
Development Workflow¶
Daily Development¶
# 1. Activate environment
conda activate syndi # or claire
# 2. Pull latest changes
git pull
# 3. Update dependencies if needed
cd frontend && npm install
conda env update -f environment.yml --prune
# 4. Start development servers
make start-dev ENV=dev ORG=myorg
# 5. Make changes (auto-reload handles updates)
# 6. Test changes
make test-all
# 7. Commit
git add .
git commit -m "Your changes"
git push
Production Simulation¶
Test packaged code locally before AWS deployment:
# Must use syndi environment (Python 3.9)
conda activate syndi
# Build and serve Lambda
make serve-lambda ENV=dev ORG=myorg
# In another terminal, serve frontend
make serve-webapp ENV=dev ORG=myorg
Troubleshooting¶
Import Errors After Environment Switch¶
Symptom: ModuleNotFoundError after switching environments
Solution:
# Clean and rebuild backend
make clean-backend
conda activate syndi
make build-backend ENV=dev ORG=myorg
Missing Dependencies¶
Symptom: Import errors for specific packages
Solution:
# Reinstall from requirements
conda activate syndi
pip install -r backend/layers/dependencies/requirements.txt
# Or reinstall specific package
pip install <package-name>
Environment Verification¶
Check which Python is being used:
which python
which pip
# Should point to syndi environment paths
# Example: /home/user/miniconda3/envs/syndi/bin/python
Wrong Environment Active¶
Symptom: Code runs but uses wrong Python version
Solution:
# Check active environment
conda env list
# Active environment has * marker
# Activate correct one
conda activate syndi
Port Already in Use¶
Symptom: “Address already in use” on port 8000 or 3000
Solution:
# Stop all servers
make stop-all
# Or manually
pkill -f "uvicorn.*rawscribe"
pkill -f "vite"
# Check what's using ports
lsof -i :8000 # Backend
lsof -i :3000 # Frontend
lsof -i :5173 # Vite alternate port
Configuration Not Loading¶
Symptom: “Config file not found” errors
Solution:
# Redeploy configuration
make clean-config
make config ENV=dev ORG=myorg
# Verify configs exist
ls -la backend/rawscribe/.config/config.json
ls -la frontend/public/config.json
Playwright Browser Issues¶
Symptom: E2E tests fail with browser errors
Solution:
# Reinstall Playwright browsers
cd frontend
npx playwright install
# Install system dependencies (Linux)
npx playwright install-deps
Project Structure¶
Understanding the project layout:
2CLAIRE/
├── backend/ # Python backend
│ ├── rawscribe/ # Lambda application code
│ │ ├── main.py # Lambda handler
│ │ ├── routes/ # API endpoints
│ │ ├── utils/ # Utilities (auth, config, etc.)
│ │ └── .config/ # Generated config (not in git)
│ ├── layers/ # Lambda layers
│ │ └── dependencies/ # Python dependencies
│ ├── tests/ # Backend tests
│ └── .build/ # Build artifacts (not in git)
├── frontend/ # React frontend
│ ├── src/ # Source code
│ │ ├── components/ # React components
│ │ ├── pages/ # Page components
│ │ └── shared/ # Shared utilities
│ ├── public/ # Static assets
│ │ └── config.json # Generated config (not in git)
│ ├── tests/ # Frontend tests
│ └── dist/ # Build output (not in git)
├── infra/ # Infrastructure
│ ├── .config/ # Configs (not in git)
│ │ ├── lambda/ # Backend configs
│ │ └── webapp/ # Frontend configs
│ ├── example-.config/ # Config templates (in git)
│ └── scripts/ # Deployment scripts
├── .local/ # Local "S3" storage (not in git)
├── .aws-sam-*/ # SAM build dirs (not in git)
├── template.yaml # SAM/CloudFormation template
├── Makefile # Build automation
└── environment.yml # Conda environment spec
Next Steps¶
After completing local setup:
Read: Local Development Workflow - Daily development tasks
Read: Testing Guide - How to test your changes
Try: Configuration System - Understanding configs
Deploy: Makefile Deployment - Deploy to AWS