René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"[Feedback request] Experimental Python 3.14 OCI images with thread-based multi-concurrency for Lambda Managed Instances","articleBody":"\u003e ⚠️ **Experimental - not for production use.** These images are provided for feedback and testing only. They should not be used for production workloads. Functions based on these images are not covered by AWS Technical Support and are not eligible for the Lambda SLA. These images may be modified or discontinued without notice.\n\nWe have published experimental Python OCI images that add **thread-based multi-concurrency** as an opt-in alternative to the default process-based model on [Lambda Managed Instances (LMI)](https://docs.aws.amazon.com/lambda/latest/dg/lambda-managed-instances.html). These images let you choose how your function handles concurrent requests: via processes, threads, or a hybrid of both.\n\n\u003e **Note:** Thread and hybrid concurrency modes require Lambda Managed Instances. On standard (on-demand) Lambda, multi-concurrency is not supported. The image behaves identically to the production-ready `python:3.14` image.\n\n**Images available now:**\n\n```\npublic.ecr.aws/lambda/python:3.14-experimental            # multi-arch manifest\npublic.ecr.aws/lambda/python:3.14-experimental-x86_64\npublic.ecr.aws/lambda/python:3.14-experimental-arm64\n```\n\n\n---\n\n### Why thread-based concurrency?\n\nThe default process model on LMI (documented at [Python runtime for Lambda Managed Instances](https://docs.aws.amazon.com/us_en/lambda/latest/dg/lambda-managed-instances-python-runtime.html)) spawns N separate Python processes. This is safe and simple, but has limitations:\n\n| Problem | Impact |\n|---------|--------|\n| Memory multiplication | N processes × handler memory (e.g., 16 × 300MB = 4.8GB) |\n| Duplicate initialization | Each process imports handler independently |\n| No shared in-memory state | Caches, models, connections cannot be shared between requests |\n\nThread mode addresses these by running N concurrent requests as threads within a single process, sharing memory, handler code, and any module-level state.\n\n---\n\n### What's different from the production-ready `python:3.14` image?\n\n| Aspect | Production-ready `python:3.14` | Experimental `python:3.14-experimental` |\n|--------|------------------------|------------------------------------------|\n| Production-ready? | Yes | **No - experimental, not for production workloads** |\n| Concurrency model | Process-only (N processes × 1 thread) | Process, Thread, or Hybrid (configurable) |\n| `AWS_LAMBDA_CONCURRENCY_MODE` env var | Not recognized | Accepts `process` / `thread` / `hybrid` |\n| Pre-fork hooks (`@register_pre_fork`) | Not available | Available for process/hybrid modes |\n| Default behavior (no env var set) | N processes × 1 thread | **Identical** to production-ready image |\n| Python binary | Python 3.14 (standard, GIL-enabled) | Python 3.14 (standard, GIL-enabled) |\n\n\u003e **Important:** If you do NOT set `AWS_LAMBDA_CONCURRENCY_MODE`, the experimental image behaves **exactly like the production-ready image**. Thread mode is strictly opt-in.\n\n### Things to know before opting in (thread and hybrid modes)\n\n#### Your code needs to be thread-safe\n\nThread and hybrid modes run concurrent requests as threads within a single process. This means all threads share memory, module-level objects, and file descriptors. Code that works correctly in process mode (where each request gets its own isolated process) may need changes:\n\n- **Shared mutable state**: All threads share process memory. Patterns like shared `boto3.Session()`, shared `boto3.resource()`, `requests.Session` with header mutation, writing to `/tmp` with fixed filenames, and `global_counter += 1` will break. Use `threading.local()`, `threading.Lock()`, or per-request file paths as appropriate.\n- **Database connections**: Connections and sockets opened at module level are shared across threads. Use connection pools or per-thread connections. (Pre-created `boto3.client()` operations are safe; the client handles concurrency internally.)\n\n#### Known platform limitations\n\nThe following observability features do not fully work in thread/hybrid modes yet. These are temporary limitations of the experimental image. The production-ready release will include full trace propagation and recursion detection support.\n\n- **Trace ID propagation**: Downstream AWS service calls won't carry your Lambda invocation's trace ID. Traces appear disconnected in X-Ray.\n- **Recursion detection**: The platform can't detect and stop recursive loops (e.g., Lambda → SQS → Lambda). Use `process` mode for these workloads.\n- **Custom segment generation (Application Signals / ADOT)**: The [one-click Application Signals](https://docs.aws.amazon.com/lambda/latest/dg/monitoring-application-signals.html) console feature, ADOT auto-instrumentation, and custom X-Ray subsegments won't produce connected traces.\n\n---\n\n### How to use it\n\n#### 1. Pull the experimental image\n\n```dockerfile\nFROM public.ecr.aws/lambda/python:3.14-experimental\n\nCOPY requirements.txt .\nRUN pip install -r requirements.txt\n\nCOPY app.py ${LAMBDA_TASK_ROOT}\nCMD [\"app.handler\"]\n```\n\n#### 2. Configure concurrency mode\n\nN is the maximum number of concurrent requests routed to a single execution environment, set by your function's [`PerExecutionEnvironmentMaxConcurrency`](https://docs.aws.amazon.com/lambda/latest/dg/lambda-managed-instances-python-runtime.html#lambda-managed-instances-python-concurrency-config) configuration.\n\nSet the `AWS_LAMBDA_CONCURRENCY_MODE` environment variable on your Lambda function or in your Dockerfile (`ENV AWS_LAMBDA_CONCURRENCY_MODE=thread`):\n\n| Mode | Environment Variable | What happens |\n|------|---------------------|--------------|\n| **Process** (default) | `AWS_LAMBDA_CONCURRENCY_MODE=process` | N processes × 1 thread each. Identical to production-ready image. |\n| **Thread** | `AWS_LAMBDA_CONCURRENCY_MODE=thread` | 1 process × N threads. All threads share memory and handler. |\n| **Hybrid** | `AWS_LAMBDA_CONCURRENCY_MODE=hybrid` | K processes × M threads. K = allocated vCPUs, M = floor(N / K). |\n\n\u003e **Note (hybrid mode):** M is calculated as `floor(N / K)`. If N doesn't divide evenly by K, some concurrency slots may be unused. For example, N=16 on 3 vCPUs → 3 processes × 5 threads = 15 concurrent handlers (1 slot unused).\n\n#### 3. Example: Thread mode with shared S3 download\n\n```python\nimport boto3\nimport threading\nimport json\nimport os\n\n# Module-level initialization runs ONCE (before threads start)\n# This is the key advantage: expensive setup happens once, shared by all threads\n\ns3 = boto3.client('s3')  # Pre-created client is thread-safe for operations\n_lock = threading.Lock()\n_shared_data = None\n\ndef _load_data():\n    \"\"\"Download large file from S3 once, share across all threads.\"\"\"\n    global _shared_data\n    if _shared_data is None:\n        with _lock:\n            if _shared_data is None:  # Double-checked locking\n                response = s3.get_object(Bucket='my-bucket', Key='large-config.json')\n                _shared_data = json.loads(response['Body'].read())\n    return _shared_data\n\ndef handler(event, context):\n    # All concurrent threads share the same _shared_data\n    data = _load_data()\n    result = process(event, data)\n    return {\"statusCode\": 200, \"body\": json.dumps(result)}\n```\n\n#### 4. Example: Hybrid mode\n\nHybrid mode automatically calculates K (processes) from your allocated vCPUs and M (threads per process) from `floor(N / K)`. This gives you a balance of fault isolation and memory sharing.\n\n```bash\n# On a 4-vCPU instance with N=16 (PerExecutionEnvironmentMaxConcurrency):\n# hybrid → 4 processes × 4 threads = 16 concurrent handlers\nAWS_LAMBDA_CONCURRENCY_MODE=hybrid\n```\n\nEach process independently loads the handler, then spawns M threads that share that process's memory.\n\n---\n\n### Pre-fork hooks (`@register_pre_fork`)\n\nIn **process** and **hybrid** modes, each child process re-imports the handler independently (separate INIT per process). Pre-fork hooks let you run setup **once in the parent** before workers spawn. This is mainly useful for starting inference servers, downloading model files to `/tmp`, or other one-time setup that all children can benefit from.\n\n```python\nimport subprocess\nfrom awslambdaric.lambda_concurrency_hooks import register_pre_fork\n\n@register_pre_fork\ndef start_inference_server():\n    \"\"\"Start model server once; all worker processes send requests to it.\"\"\"\n    subprocess.Popen([\"python\", \"serve.py\", \"--port\", \"8000\"])\n```\n\n**Notes:**\n- Only fires in process/hybrid modes. In thread mode, use module-level code instead (runs once before threads start).\n- In hybrid mode, pre-fork hooks only execute when vCPU count \u003e 1 (otherwise `num_processes = 1`, which behaves exactly like thread mode).\n- Children are spawned with `multiprocessing.get_context(\"spawn\")`. Each child gets a fresh Python interpreter with no inherited objects or file descriptors. Pre-fork hooks are for setup tasks, not for sharing in-memory state.\n\n---\n\n### Compatibility\n\n| Environment | Works? | Notes |\n|-------------|:---:|-------|\n| Lambda Managed Instances (LMI) | ✅ | Full thread/hybrid support with `PerExecutionEnvironmentMaxConcurrency` |\n| AWS SAM CLI `sam local invoke` | ✅ | Same as production-ready image |\n| Standard Lambda (non-LMI) | ⚠️ | Image works but **multi-concurrency modes (thread/hybrid) are not supported**. Behaves identically to `python:3.14`. |\n\n---\n\n### We want your feedback!\n\nWe'd love to hear from you:\n\n1. **What's your use case?** (I/O-bound API calls, ML inference, data processing, etc.)\n2. **Which mode are you using?** (thread, hybrid, process)\n3. **Are you using pre-fork hooks?** If so, what for?\n4. **What libraries are you using with thread mode?** Did anything break?\n5. **Memory/performance comparisons** - did thread mode reduce your memory footprint vs process mode?\n6. **Would Copy-on-Write (CoW) memory sharing between processes be useful for your workload?** (Currently, each child process re-imports independently with no shared memory from parent.)\n7. **Any issues, unexpected behavior, or suggestions for improvement?**\n\nPlease share your experience in the comments below. Your feedback directly shapes whether and how this ships to the production-ready images.\n\n---\n\n### Links\n\n- [Lambda Managed Instances overview](https://docs.aws.amazon.com/lambda/latest/dg/lambda-managed-instances.html)\n- [Python runtime for LMI](https://docs.aws.amazon.com/us_en/lambda/latest/dg/lambda-managed-instances-python-runtime.html) (current process-based docs)\n- [LMI Core Concepts](https://docs.aws.amazon.com/lambda/latest/dg/lambda-managed-instances-core-concepts.html)\n- [Getting started with LMI](https://docs.aws.amazon.com/lambda/latest/dg/lambda-managed-instances-getting-started.html)\n- [LMI Scaling](https://docs.aws.amazon.com/lambda/latest/dg/lambda-managed-instances-scaling.html)\n- [Understanding the LMI execution environment](https://docs.aws.amazon.com/lambda/latest/dg/lambda-managed-instances-execution-environment.html)\n","author":{"url":"https://github.com/vip-amzn","@type":"Person","name":"vip-amzn"},"datePublished":"2026-07-20T13:57:07.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":0},"url":"https://github.com/215/aws-lambda-python-runtime-interface-client/issues/215"}