euca-import-volume conversion stuck on pending

Question

I'm following imaging guide docs to create an EBS backed image: https://docs.eucalyptus.cloud/eucalyptus/4.4.5/index.html#image-guide/img_task_install_hvm_image.html

#import to volume
euca-import-volume CentOS-7-x86_64-GenericCloud-1901.raw --format raw --availability-zone zone-1 --bucket imagebucket --description "ebs backed centos7"

The conversion task after import is stuck in Pending conversion at ApproximateBytesConverted 0 and is making no progress. Any ideas on common causes of this?

[root@desktop ~]# euca-describe-conversion-tasks
TaskType    IMPORTVOLUME    TaskId  import-vol-a6baf98c ExpirationTime  Wed Apr 17 09:25:47 EDT 2019    Status  active  StatusMessage   Pending conversion
DISKIMAGE   DiskImageFormat RAW DiskImageSize   8589934592  VolumeId    vol-4ddda573    VolumeSize  8   AvailabilityZone    zone-1  ApproximateBytesConverted   0

The imaging worker is configured with m2.2xlarge but I'm seeing conflicting info about whether its launching.

[root@cloud ~]# euca-describe-instances --filter tag-value=euca-internal-imaging-workers
[root@cloud ~]# esi-describe-images
    SERVICE     VERSION  ACTIVE     IMAGE      INSTANCES 
    imaging     4.4.101    *     emi-ebfa1114      1     
 loadbalancing  4.4.101    *     emi-ebfa1114      0   

Answer 1

The conversion task staying in the pending state means there is an issue with the imaging worker instance.

The reason you do not see output from euca-describe-instances is that the imaging worker runs in the (eucalyptus)imaging service account rather than the eucalyptus account. To see the instance details:

#
# euare-rolelistbypath --as-account '(eucalyptus)imaging'
arn:aws:iam::123456789012:role/internal/imaging/euca-internal-imaging-service-Role-XXXXXXXXXXXXX
arn:aws:iam::123456789012:role/imaging/ImagingServiceAdministrator
# 
#
# eval $(euare-assumerole arn:aws:iam::123456789012:role/imaging/ImagingServiceAdministrator)
# 
#
# euca-describe-instances
RESERVATION r-3170...
# 
#
# eval $(euare-releaserole)
#
#

If there are issues with launching the instance then the CloudFormation service may have useful details, eg:

# euform-describe-stacks
# euform-describe-stack-events

The most common issue with the imaging worker is forgetting to set the ntp server to use, eg:

# euctl services.imaging.worker.ntp_server=time.google.com

this is just an example, you should use the ntp server for your environment.

To allow you to SSH to the imaging worker instance and check logs you should use euca-import-keypair as the imaging service administrator and then configure the key name:

# euctl services.imaging.worker.keyname=KEYNAMEHERE

An easy way to restart the imaging worker and pick up any configuration changes is:

# euctl services.imaging.worker.configured=false
# # wait for shutdown ...
# euctl services.imaging.worker.configured=true

Answer 2

I was able to use the manual helper instance method to create EBS backed hvm images: https://docs.eucalyptus.cloud/eucalyptus/4.4.5/index.html#image-guide/img_task_install_hvm_image.html

Option C: Install an EBS image using a helper instance

It is technically a supported method so I don't feel like its a workaround but it would be convenient to understand why Option A (euca-import-volume) isn't working for me as described.