Aller au contenu principal

Python Service Guidelines

Logging​

CRITICAL RULE: All Python services MUST implement comprehensive logging with structured error information. Logging is essential for debugging production issues and understanding service behavior.

Logger Configuration​

Location: python-service/main.py

Setup:

  • Configure logging at application startup with appropriate level (DEBUG in development, INFO in production)
  • Use Python's standard logging module
  • Configure format: "%(asctime)s - %(name)s - %(levelname)s - %(message)s"

Example:

import logging

# Configure logging
logging.basicConfig(
    level=logging.DEBUG if settings.DEBUG else logging.INFO,
    format="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
)
logger = logging.getLogger(__name__)

Logger Usage in Services​

CRITICAL RULE: Always use module-level loggers - Create a logger at the top of each module using logging.getLogger(__name__).

Example:

import logging

logger = logging.getLogger(__name__)

class MyService:
    async def my_method(self):
        logger.info("Processing request...")

Log Levels​

Use appropriate log levels:

  • logger.debug(): Detailed diagnostic information (development only)

    • Function entry/exit
    • Parameter values
    • Intermediate computation steps
    • Cache hits/misses

  • logger.info(): General informational messages

    • Successful operations
    • Service initialization
    • Important state changes
    • Request processing start/completion

  • logger.warning(): Warning messages for potentially problematic situations

    • Deprecated feature usage
    • Configuration issues
    • Performance concerns
    • Recoverable errors

  • logger.error(): Error messages for failures

    • Failed operations
    • External service errors
    • Data validation failures
    • MUST include full traceback (see Error Logging section)

Error Logging​

CRITICAL RULE: All error logs MUST include complete traceback information. Use traceback.format_exc() to capture full stack traces.

Required Information in Error Logs:

  1. Error message: Clear description of what failed
  2. Full traceback: Complete stack trace using traceback.format_exc()
  3. Context data: Relevant request parameters, IDs, sizes, etc.
  4. Operation details: What operation was being performed

Example - Router Error Logging:

import logging
import traceback

logger = logging.getLogger(__name__)

@router.post("/html-to-pdf", response_class=Response)
async def html_to_pdf(request: HtmlToPdfRequest):
    try:
        pdf_service = get_pdf_service()
        pdf_bytes = await pdf_service.html_to_pdf(request.html, request.css)
        # ... return response
    except Exception as e:
        error_traceback = traceback.format_exc()
        logger.error(
            f"Failed to generate PDF from HTML: {e}\n"
            f"HTML length: {len(request.html)} chars\n"
            f"CSS provided: {bool(request.css)}\n"
            f"Traceback:\n{error_traceback}"
        )
        raise HTTPException(
            status_code=500,
            detail=f"PDF generation failed: {str(e)}",
        )

Example - Service Error Logging:

import logging
import traceback

logger = logging.getLogger(__name__)

class PdfService:
    async def html_to_pdf(
        self,
        html: str,
        css: Optional[str] = None,
        base_url: Optional[str] = None,
        landscape: bool = False,
    ) -> bytes:
        try:
            logger.debug(f"Creating HTML document from string (length: {len(html)} chars)")
            html_doc = HTML(string=html, base_url=base_url)
            # ... processing
            logger.info(f"Generated PDF from HTML: {len(pdf_bytes)} bytes | Landscape: {landscape}")
            return pdf_bytes
        except Exception as e:
            error_traceback = traceback.format_exc()
            logger.error(
                f"Failed to generate PDF from HTML: {e}\n"
                f"HTML length: {len(html)} chars\n"
                f"HTML preview (first 500 chars): {html[:500]}\n"
                f"CSS provided: {bool(css)}\n"
                f"Base URL: {base_url}\n"
                f"Landscape: {landscape}\n"
                f"Traceback:\n{error_traceback}"
            )
            raise

Contextual Information​

CRITICAL RULE: Always include relevant context in logs to help debug issues:

For Requests:

  • Request parameters (IDs, sizes, types)
  • User context (if available)
  • Operation type
  • Resource identifiers

For Operations:

  • Input sizes (HTML length, file sizes, etc.)
  • Configuration values (base URLs, flags)
  • Intermediate results (when relevant)
  • Performance metrics (durations, sizes)

Example:

logger.info(
    f"Processing document conversion → "
    f"DocumentID: {document_id} | "
    f"Format: {format} | "
    f"Size: {len(content)} bytes"
)

Debug Logging​

When to use logger.debug():

  • Function entry/exit points
  • Parameter values (non-sensitive)
  • Intermediate computation steps
  • Cache operations (hits/misses)
  • External API call details (request/response sizes)

Example:

logger.debug(f"Creating HTML document from string (length: {len(html)} chars)")
logger.debug(f"Adding custom CSS (length: {len(css)} chars)")
logger.debug(f"Generating PDF with {len(stylesheets)} stylesheet(s)")

Service Initialization Logging​

CRITICAL RULE: Log service initialization with version information for debugging compatibility issues.

Example:

@asynccontextmanager
async def lifespan(app: FastAPI) -> AsyncGenerator[None, None]:
    logger.info("🚀 Starting Python Services...")
    logger.info("📦 Loading models...")
    logger.info("📄 PDF generation: WeasyPrint")
    
    # Log versions for debugging
    try:
        import weasyprint
        import pydyf
        logger.info(f"WeasyPrint version: {weasyprint.__version__}")
        logger.info(f"pydyf version: {pydyf.__version__}")
    except ImportError as e:
        logger.warning(f"Could not import WeasyPrint/pydyf to check versions: {e}")
    
    yield
    
    logger.info("👋 Shutting down Python Services...")

Best Practices​

  1. Always log errors with full context: Include traceback, parameters, and operation details
  2. Use appropriate log levels: Don't use ERROR for warnings, don't use INFO for debug information
  3. Include relevant metrics: Sizes, durations, counts help identify performance issues
  4. Log service versions: Helps debug compatibility issues (e.g., WeasyPrint/pydyf versions)
  5. Structured logging: Use consistent format across all logs
  6. Don't log sensitive data: Never log passwords, tokens, or personal information
  7. Log at operation boundaries: Log at the start and completion of important operations
  8. Include request identifiers: When available, include request IDs or user IDs for tracing

Error Response Format​

CRITICAL RULE: Error responses MUST include detailed error messages while keeping HTTP responses user-friendly.

Pattern:

  • Log: Full error details with traceback (for debugging)
  • HTTP Response: User-friendly error message (without sensitive details)

Example:

except Exception as e:
    error_traceback = traceback.format_exc()
    # Log full details for debugging
    logger.error(
        f"Failed to generate PDF: {e}\n"
        f"Traceback:\n{error_traceback}"
    )
    # Return user-friendly message
    raise HTTPException(
        status_code=500,
        detail=f"PDF generation failed: {str(e)}",
    )

Logging Checklist​

When adding logging to a new service or endpoint:

  • Logger created at module level: logger = logging.getLogger(__name__)
  • All exceptions caught and logged with traceback.format_exc()
  • Context information included (IDs, sizes, parameters)
  • Appropriate log levels used (debug/info/warning/error)
  • Service initialization logged with version info
  • Operation start/completion logged
  • No sensitive data in logs
  • Error responses are user-friendly (detailed info only in logs)

Error Handling​

CRITICAL RULE: All exceptions MUST be caught, logged with full context, and re-raised or converted to HTTP exceptions.

Exception Handling Pattern​

In Routers:

@router.post("/endpoint")
async def my_endpoint(request: MyRequest):
    try:
        result = await service.process(request)
        return result
    except ValueError as e:
        logger.warning(f"Validation error: {e}")
        raise HTTPException(status_code=400, detail=str(e))
    except Exception as e:
        error_traceback = traceback.format_exc()
        logger.error(
            f"Unexpected error in my_endpoint: {e}\n"
            f"Request: {request.model_dump()}\n"
            f"Traceback:\n{error_traceback}"
        )
        raise HTTPException(
            status_code=500,
            detail="Internal server error",
        )

In Services:

async def process(self, request: MyRequest) -> Result:
    try:
        # Processing logic
        return result
    except SpecificError as e:
        logger.error(f"Specific error occurred: {e}")
        raise  # Re-raise to let router handle HTTP response
    except Exception as e:
        error_traceback = traceback.format_exc()
        logger.error(
            f"Unexpected error in process: {e}\n"
            f"Request data: {request}\n"
            f"Traceback:\n{error_traceback}"
        )
        raise

Code Style​

  • Follow PEP 8 style guide
  • Use type hints for all function parameters and return types
  • Use async/await for all I/O operations
  • Keep functions focused and small (< 100 lines when possible)
  • Use descriptive variable and function names
  • Add docstrings for all public functions and classes

Dependencies​

CRITICAL RULE: Always pin dependency versions in requirements.txt to ensure reproducible builds and avoid compatibility issues.

Example:

weasyprint==62.3    # PDF generation (CSS3 support)
pydyf==0.11.0       # PDF low-level library (required by WeasyPrint 62.3, 0.12.1 has bug)

When adding new dependencies:

  1. Pin exact version: package==1.2.3
  2. Add comment explaining why the dependency is needed
  3. Document any version constraints or known issues
  4. Test compatibility with existing dependencies

FastAPI Best Practices​

  • Use Pydantic models for request/response validation
  • Use dependency injection for services
  • Return appropriate HTTP status codes
  • Use response models for type safety
  • Document endpoints with docstrings

Testing​

  • Write unit tests for all service methods
  • Test error cases, not just happy paths
  • Mock external dependencies
  • Use pytest for testing framework
  • Aim for >80% code coverage