前端之家

  1. 首页
  2. 问答

AWS ECS容器卡在重启循环中

• 问答

尝试启动我的第一个ECS集群。 容器已成功启动，但每分钟左右都会重新启动。 群集服务正在使用应用程序负载平衡器。

任务定义：

{
  "containerDefinitions": [
    {
      "dnsSearchDomains": null,"entryPoint": null,"portMappings": [
        {
          "hostPort": 80,"protocol": "tcp","containerPort": 80
        }
      ],"command": [
        "bash","deploy.sh"
      ],"cpu": 512,"memory": 1961,"memoryReservation": 256,"image": "1","essential": true,}
  ],"placementConstraints": [],"memory": null,"taskRoleArn": null,"compatibilities": [
    "EC2"
  ],"taskDefinitionArn": "arn:aws:ecs:...","family": "service","networkMode": "awsvpc","status": "actIVE","inferenceaccelerators": null,"proxyConfiguration": null,"volumes": []
}

ecs代理日志（已删除和重命名的详细信息），每分钟左右都会经历以下循环

00:05Z [INFO] Handling ENI attachment
00:05Z [INFO] Starting ENI ack timer 
00:05Z [INFO] Adding task eni attachment 
00:05Z [INFO] Emitting task ENI attached event for: ENI Attachment: 
00:05Z [INFO] TaskHandler: Adding event: TaskChange: arn:... -> NONE,ENI Attachment:  sent: false
00:05Z [INFO] TaskHandler: Sending task attachment change: TaskChange: arn:... -> NONE,ENI Attachment:  sent: false
00:11Z [INFO] Managed task arn:... unable to create state change event for container  create container state change event api: status not recognized by ECS: NONE
00:11Z [INFO] Managed task arn:... unable to create state change event for container [~internal~ecs~pause]: create container state change event api: status not recognized by ECS: NONE
00:11Z [INFO] Managed task arn:... unable to create task state change event []: create task state change event api: status not recognized by ECS: NONE
00:11Z [INFO] Managed task arn:... waiting for any previous stops to complete. Sequence number: 110
00:11Z [INFO] Managed task arn:... waiting for event for task
00:11Z [INFO] Managed task arn:... no longer waiting
00:11Z [INFO] Managed task arn:... wait over; ready to move towards status: RUNNING
00:11Z [INFO] Managed task arn:... task not steady state or terminal; progressing it
00:11Z [INFO] Managed task arn:... waiting for event for task
00:11Z [INFO] Managed task arn:... got container [~internal~ecs~pause] event: [PULLED]
00:11Z [INFO] Managed task arn:... handling container change [{PULLED { <nil> [] <nil> [] map[]  {UNKNOWN <nil> 0 }  <nil>} ContainerStatusChangeEvent}] for container [~internal~ecs~pause]
00:11Z [INFO] Managed task arn:... unable to create state change event for container [~internal~ecs~pause]: create container state change event api: status not recognized by ECS: PULLED
00:11Z [INFO] Managed task arn:... task not steady state or terminal; progressing it
00:11Z [INFO] Managed task arn:... waiting for event for task
00:11Z [INFO] Task engine arn:... creating container: ~internal~ecs~pause
00:11Z [INFO] Task engine arn:... created container name mapping for task:  ~internal~ecs~pause -> 
00:11Z [INFO] Creating cgroup 
00:11Z [INFO] Managed task arn:... transitioned resource [cgroup] to [CREATED]
00:11Z [INFO] Managed task arn:... got resource [cgroup] event: [CREATED]
00:11Z [INFO] Managed task arn:... task not steady state or terminal; progressing it
00:11Z [INFO] Managed task arn:... waiting for event for task
00:11Z [INFO] Task engine arn:... created docker container for task: ~internal~ecs~pause -> ...
00:11Z [INFO] Task engine arn:... created docker container for task: ~internal~ecs~pause -> ...,took 103.387592ms
00:11Z [INFO] Managed task arn:... got container [~internal~ecs~pause] event: [CREATED]
00:11Z [INFO] Managed task arn:... handling container change [{CREATED {... <nil> [] <nil> []  com.amazonaws.ecs.container-name:~internal~ecs~pause com.amazonaws.ecs.task-arn:arn:aws:ecs:...  com.amazonaws.ecs.task-definition-version:15]    {UNKNOWN <nil> 0 } none 0xc00011ab00} ContainerStatusChangeEvent}] for container [~internal~ecs~pause]
00:11Z [INFO] Managed task arn:... unable to create state change event for container [~internal~ecs~pause]: create container state change event api: status not recognized by ECS: CREATED
00:11Z [INFO] Managed task arn:... task not steady state or terminal; progressing it
00:11Z [INFO] Managed task arn:... waiting for event for task
00:11Z [INFO] Task engine arn:... starting container: ~internal~ecs~pause
00:11Z [INFO] Managed task arn:... got container [~internal~ecs~pause] event: [RUNNING]
00:11Z [INFO] Managed task arn:... handling container change [{RUNNING {... <nil> [] <nil> []  com.amazonaws.ecs.container-name:~internal~ecs~pause com.amazonaws.ecs.task-arn:arn:aws:ecs:...  com.amazonaws.ecs.task-definition-version:15]    {UNKNOWN <nil> 0 } none 0xc00011b600} ContainerStatusChangeEvent}] for container [~internal~ecs~pause]
00:11Z [INFO] Managed task arn:... unable to create state change event for container [~internal~ecs~pause]: create container state change event api: status not recognized by ECS: RUNNING
00:11Z [INFO] Managed task arn:... task not steady state or terminal; progressing it
00:11Z [INFO] Managed task arn:... waiting for event for task
00:11Z [INFO] Task engine arn:... setting up container resources for container [~internal~ecs~pause]
00:11Z [INFO] Task engine arn:... started docker container for task: ~internal~ecs~pause -> ...,took 221.148618ms
00:11Z [INFO] Managed task arn:... got container [~internal~ecs~pause] event: [RUNNING]
00:11Z [INFO] Managed task arn:... handling container change [{RUNNING {... <nil> [] <nil> []  com.amazonaws.ecs.container-name:~internal~ecs~pause com.amazonaws.ecs.task-arn:arn:aws:ecs:...  com.amazonaws.ecs.task-definition-version:15]    {UNKNOWN <nil> 0 } none 0xc0007aed00} ContainerStatusChangeEvent}] for container [~internal~ecs~pause]
00:11Z [INFO] Managed task arn:... redundant container state change. ~internal~ecs~pause to RUNNING,but already RUNNING
00:11Z [INFO] Managed task arn:... task not steady state or terminal; progressing it
00:11Z [INFO] Managed task arn:... waiting for event for task
00:11Z [INFO] Task engine arn:... associated with ip address '1.2.3.4'
00:11Z [INFO] Managed task arn:... got container [~internal~ecs~pause] event: [RESOURCES_PROVISIONED]
00:11Z [INFO] Managed task arn:... handling container change [{RESOURCES_PROVISIONED {... <nil> [] <nil> [] map[]  {UNKNOWN <nil> 0 }  <nil>} ContainerStatusChangeEvent}] for container [~internal~ecs~pause]
00:11Z [INFO] Managed task arn:... unable to create state change event for container [~internal~ecs~pause]: create container state change event api: internal container: ~internal~ecs~pause
00:11Z [INFO] Managed task arn:... task not steady state or terminal; progressing it
00:11Z [INFO] Managed task arn:... waiting for event for task
00:11Z [INFO] Task engine arn:... pulling image  for container  concurrently
00:11Z [INFO] Task engine arn:... recording timestamp for starting image pulltime:  
00:11Z [INFO] Updating container reference  in Image State - 
00:11Z [INFO] Task engine arn:... finished pulling image  for container  in 99.366882ms
00:11Z [INFO] Managed task arn:... got container  event: [PULLED]
00:11Z [INFO] Managed task arn:... handling container change [{PULLED { <nil> [] <nil> [] map[]  {UNKNOWN <nil> 0 }  <nil>} ContainerStatusChangeEvent}] for container 
00:11Z [INFO] Managed task arn:... unable to create state change event for container  create container state change event api: status not recognized by ECS: PULLED
00:11Z [INFO] Managed task arn:... task not steady state or terminal; progressing it
00:11Z [INFO] Managed task arn:... waiting for event for task
00:11Z [INFO] Task engine arn:... creating container: 
00:11Z [INFO] Task engine arn:... created container name mapping for task:   -> 
00:12Z [INFO] Task engine arn:... created docker container for task:  -> ...
00:12Z [INFO] Task engine arn:... created docker container for task:  -> ...,took 81.682728ms
00:12Z [INFO] Managed task arn:... got container  event: [CREATED]
00:12Z [INFO] Managed task arn:... handling container change [{CREATED {... <nil> [] <nil> []   com.amazonaws.ecs.task-arn:arn:aws:ecs:...  com.amazonaws.ecs.task-definition-version:15]    {UNKNOWN <nil> 0 } container: 0xc00062eb00} ContainerStatusChangeEvent}] for container 
00:12Z [INFO] Managed task arn:... unable to create state change event for container  create container state change event api: status not recognized by ECS: CREATED
00:12Z [INFO] api/task: Updating task's known status to: CREATED,task:  arn:aws:ecs:... TaskStatus: (NONE->RUNNING) Containers:  (CREATED->RUNNING),~internal~ecs~pause (RESOURCES_PROVISIONED->RESOURCES_PROVISIONED),] ENIs: [eni id:eni-1 mac: aaa hostname: hostname ipv4addresses: [1.2.3.4],ipv6addresses: [],dns: [],dns search: [],gateway ipv4: [1.2.3.4/20][,VLan ID: [],TrunkinterfaceMacAddress: []],]
00:12Z [INFO] Managed task arn:... container change also resulted in task change  [RUNNING]
00:12Z [INFO] Managed task arn:... unable to create task state change event []: create task state change event api: status not recognized by ECS: CREATED
00:12Z [INFO] Managed task arn:... task not steady state or terminal; progressing it
00:12Z [INFO] Managed task arn:... waiting for event for task
00:12Z [INFO] Task engine arn:... starting container: 
00:12Z [INFO] Managed task arn:... got container  event: [RUNNING]
00:12Z [INFO] Managed task arn:... handling container change [{RUNNING {... <nil> [] <nil> []   com.amazonaws.ecs.task-arn:arn:aws:ecs:...  com.amazonaws.ecs.task-definition-version:15]    {UNKNOWN <nil> 0 } container: 0xc0007aeb00} ContainerStatusChangeEvent}] for container 
00:12Z [INFO] Managed task arn:... sending container change event  arn:aws:ecs:...  -> RUNNING,Known Sent: NONE
00:12Z [INFO] Managed task arn:... sent container change event  arn:aws:ecs:...  -> RUNNING,Known Sent: NONE
00:12Z [INFO] api/task: Updating task's known status to: RUNNING,task:  arn:aws:ecs:... TaskStatus: (CREATED->RUNNING) Containers:  (RUNNING->RUNNING),]
00:12Z [INFO] Managed task arn:... container change also resulted in task change  [RUNNING]
00:12Z [INFO] Managed task arn:... sending task change event arn:... -> RUNNING,Known Sent: NONE,PullStartedAt:   PullStoppedAt:   ExecutionStoppedAt: ]
00:12Z [INFO] TaskHandler: batching container event  -> RUNNING,Known Sent: NONE
00:12Z [INFO] TaskHandler: Adding event: TaskChange: arn:... -> RUNNING,PullStartedAt:   PullStoppedAt:   ExecutionStoppedAt:,arn:aws:ecs:...  -> RUNNING,Known Sent: NONE] sent: false
00:12Z [INFO] TaskHandler: Sending task change: TaskChange: arn:... -> RUNNING,Known Sent: NONE] sent: false
00:12Z [INFO] Managed task arn:... sent task change event arn:... -> RUNNING,PullStartedAt:   PullStoppedAt:   ExecutionStoppedAt: ]
00:12Z [INFO] Managed task arn:... task at steady state: RUNNING
00:12Z [INFO] Managed task arn:... waiting for event for task
00:12Z [INFO] Task engine arn:... started docker container for task:  -> ...,took 204.620728ms
00:12Z [INFO] Managed task arn:... got container  event: [RUNNING]
00:12Z [INFO] Managed task arn:... handling container change [{RUNNING {... <nil> [] <nil> []   com.amazonaws.ecs.task-arn:arn:aws:ecs:...  com.amazonaws.ecs.task-definition-version:15]    {UNKNOWN <nil> 0 } container: 0xc00062f600} ContainerStatusChangeEvent}] for container 
00:12Z [INFO] Managed task arn:... redundant container state change.  to RUNNING,but already RUNNING
00:12Z [INFO] Managed task arn:... task at steady state: RUNNING
00:12Z [INFO] Managed task arn:... waiting for event for task
00:14Z [INFO] Managed task arn:... got container  event: [STOPPED]
00:14Z [INFO] Managed task arn:... handling container change [{STOPPED {... 0xc00 {UNKNOWN <nil> 0 } container: 0xc00062f800} ContainerStatusChangeEvent}] for container 
00:14Z [INFO] Task arn:... recording execution stopped time. Essential container  stopped at:  
00:14Z [INFO] Managed task arn:... sending container change event  arn:aws:ecs:...  -> STOPPED,Exit 0,Known Sent: RUNNING
00:14Z [INFO] Managed task arn:... sent container change event  arn:aws:ecs:...  -> STOPPED,Known Sent: RUNNING
00:14Z [INFO] api/task: Updating task desired status to stopped because of container:  task:  arn:aws:ecs:... TaskStatus: (RUNNING->RUNNING) Containers:  (STOPPED->RUNNING),]]
00:14Z [INFO] Managed task arn:... task not steady state or terminal; progressing it
00:14Z [INFO] Managed task arn:... waiting for event for task
00:14Z [INFO] Managed task arn:... got resource [cgroup] event: [REMOVED]
00:14Z [INFO] Task engine arn:... stopping container [~internal~ecs~pause]
00:14Z [INFO] Task engine arn:... cleaning up the network namespace
00:14Z [INFO] TaskHandler: batching container event  -> STOPPED,Known Sent: RUNNING
00:14Z [INFO] api/task: Updating task desired status to stopped because of container:  task:  arn:aws:ecs:... TaskStatus: (RUNNING->STOPPED) Containers:  (STOPPED->STOPPED),~internal~ecs~pause (RESOURCES_PROVISIONED->STOPPED),]]
00:14Z [INFO] api/task: Updating task desired status to stopped because of container: [~internal~ecs~pause]; task:  arn:aws:ecs:... TaskStatus: (RUNNING->STOPPED) Containers:  (STOPPED->STOPPED),]]
00:14Z [INFO] Managed task arn:... task not steady state or terminal; progressing it
00:14Z [INFO] Managed task arn:... waiting for event for task
00:14Z [WARN] DockerGoClient: Unable to decode stats for container ...: context canceled
00:14Z [INFO] Container ... is terminal,stopping stats collection
00:14Z [INFO] Task engine arn:... cleaned pause container network namespace
00:14Z [INFO] Managed task arn:... got container [~internal~ecs~pause] event: [STOPPED]
00:14Z [INFO] Managed task arn:... handling container change [{STOPPED {... 0xc00 {UNKNOWN <nil> 0 } none 0xc00062fb00} ContainerStatusChangeEvent}] for container [~internal~ecs~pause]
00:14Z [INFO] Managed task arn:... unable to create state change event for container [~internal~ecs~pause]: create container state change event api: internal container: ~internal~ecs~pause
00:14Z [INFO] api/task: Updating task's known status to: STOPPED,task:  arn:aws:ecs:... TaskStatus: (RUNNING->STOPPED) Containers:  (STOPPED->STOPPED),~internal~ecs~pause (STOPPED->STOPPED),]
00:14Z [INFO] api/task: Updating task desired status to stopped because of container:  task:  arn:aws:ecs:... TaskStatus: (STOPPED->STOPPED) Containers:  (STOPPED->STOPPED),]]
00:14Z [INFO] api/task: Updating task desired status to stopped because of container: [~internal~ecs~pause]; task:  arn:aws:ecs:... TaskStatus: (STOPPED->STOPPED) Containers:  (STOPPED->STOPPED),]]
00:14Z [INFO] Managed task arn:... container change also resulted in task change [~internal~ecs~pause]: [STOPPED]
00:14Z [INFO] Managed task arn:... sending task change event arn:... -> STOPPED,Known Sent: RUNNING,PullStartedAt:   PullStoppedAt:   ExecutionStoppedAt:  
00:14Z [INFO] Managed task arn:... sent task change event arn:... -> STOPPED,PullStartedAt:   PullStoppedAt:   ExecutionStoppedAt:  
00:14Z [INFO] api/task: Updating task desired status to stopped because of container:  task:  arn:aws:ecs:... TaskStatus: (STOPPED->STOPPED) Containers:  (STOPPED->STOPPED),]]
00:14Z [INFO] Managed task arn:... task has reached stopped. Waiting for container cleanup
00:14Z [INFO] Managed task arn:... waiting for event for task
00:14Z [WARN] DockerGoClient: Unable to decode stats for container ...: context canceled
00:14Z [INFO] Container ... is terminal,stopping stats collection
00:14Z [INFO] TaskHandler: Adding event: TaskChange: arn:... -> STOPPED,PullStartedAt:   PullStoppedAt:   ExecutionStoppedAt:   arn:aws:ecs:...  -> STOPPED,Known Sent: RUNNING] sent: false
00:14Z [INFO] TaskHandler: Sending task change: TaskChange: arn:... -> STOPPED,Known Sent: RUNNING] sent: false
00:14Z [INFO] Managed task arn:... IPAM releasing ip for task eni
00:14Z [INFO] Managed task arn:... got container [~internal~ecs~pause] event: [STOPPED]
00:14Z [INFO] Managed task arn:... handling container change [{STOPPED {... 0xc000489ee0 [] <nil> []  com.amazonaws.ecs.container-name:~internal~ecs~pause com.amazonaws.ecs.task-arn:arn:aws:ecs:...  com.amazonaws.ecs.task-definition-version:15]   {UNKNOWN <nil> 0 } none 0xc000505000} ContainerStatusChangeEvent}] for container [~internal~ecs~pause]
00:14Z [INFO] Managed task arn:... redundant container state change. ~internal~ecs~pause to STOPPED,but already STOPPED
00:14Z [INFO] Managed task arn:... waiting for event for task
00:14Z [INFO] Managed task arn:... got container [~internal~ecs~pause] event: [STOPPED]
00:14Z [INFO] Managed task arn:... handling container change [{STOPPED {... 0xc000072500 [] <nil> []  com.amazonaws.ecs.container-name:~internal~ecs~pause com.amazonaws.ecs.task-arn:arn:aws:ecs:...  com.amazonaws.ecs.task-definition-version:15]   {UNKNOWN <nil> 0 } none 0xc000505400} ContainerStatusChangeEvent}] for container [~internal~ecs~pause]
00:14Z [INFO] Managed task arn:... redundant container state change. ~internal~ecs~pause to STOPPED,but already STOPPED
00:14Z [INFO] Managed task arn:... waiting for event for task
00:14Z [WARN] Udev watcher event-handler: unable to send state change: udev watcher send ENI state change: eni status already sent: ENI Attachment: 
00:52Z [INFO] Handling ENI attachment
00:52Z [INFO] Starting ENI ack timer 
00:52Z [INFO] Adding task eni attachment

任何人都可以据此指出潜在原因吗？ 还有其他帮助吗？

Boney2009 回答：AWS ECS容器卡在重启循环中
暂时没有好的解决方案，如果你有好的解决方案，请发邮件至：iooj@foxmail.com
amazon-ecs
本文链接：https://www.f2er.com/3118883.html

大家都在问