Trainium - 태그 - lee's bloghttps://ken-0913.github.io/myblog/tags/trainium/Trainium - 태그 - lee's blogHugo -- gohugo.ioko-krhyeonjae0913@gmail.com (ken-0913)hyeonjae0913@gmail.com (ken-0913)Fri, 11 Sep 2026 10:00:00 +0900LLM 스터디 6주차 - Scaling LLM Inference with vLLM and AWS Trainium Workshophttps://ken-0913.github.io/myblog/posts/llm/llm-vllm-trainium-workshop/Fri, 11 Sep 2026 10:00:00 +0900ken-0913https://ken-0913.github.io/myblog/posts/llm/llm-vllm-trainium-workshop/

AWS Workshop 아키텍처 : Amazon EKS 기반 vLLM을 활용한 엔터프라이즈 규모의 대규모 언어 모델(LLM) 배포

  • 인프라 레이어: t3.2xlarge Ubuntu 22.04 EC2(개발 환경), VPC 10.0.0.0/16 + public subnet 10.0.1.0/24, SG(22/8000/8080), EKS/ECR/S3/CFN용 IAM Role
  • EKS 클러스터: K8s 1.33, VPC CNI + OIDC, managed node group neuron-trn1-2x(trn1.2xlarge, Neuron-optimized AMI, GP2 100GB, 멀티 AZ)
  • Trainium 통합: Neuron device plugin(daemonset), Neuron scheduler extension(my-scheduler), 칩당 NeuronCore-v2 2개(380 INT8 TOPS), HBM 32GB @820GB/s
  • vLLM 배포: 컨테이너 이미지 public.ecr.aws/neuron/pytorch-inference-vllm-neuronx:0.9.1-neuronx-py310-sdk2.25.0-ubuntu22.04, Init Container 패턴으로 모델 컴파일/캐싱, 대상 모델 TinyLlama-1.1B-Chat-v1.0, tensor-parallel-size=2
  • 스토리지: S3 버킷 ai-infra-summit-vllm-models-cache-{ACCOUNT_ID}(컴파일 아티팩트 캐시), S3 CSI Driver(Mountpoint) + PV/PVC(100Gi, ReadWriteMany)
  • 네트워크/Ingress: Service(LoadBalancer, 8080), NGINX Ingress Controller(경로 기반 라우팅 /)
  • 최적화 기술: Continuous batching, OpenAI 호환 API, Tensor/Pipeline Parallelism, Memory pooling, Speculative decoding
  • 모니터링: K8s 리소스 모니터링, readiness/liveness probe, Prometheus/Grafana/CloudWatch(Lab 4)

 

]]>