'content-length' error when trying to `synccopy` blob container items to a file share

[jburger]

Problem Description

First time blobxfer user here. Apologies if I'm holding it wrong!

I'm trying to copy the contents of a blob container to a file share within the same storage account.
Not sure if this is supported as I'm also having trouble with using AzCopy for the same task?

Azure blobxfer parameters output

============================================
         Azure blobxfer parameters
============================================
         blobxfer version: 1.9.3
                 platform: Linux-5.0.0-32-generic-x86_64-with-Ubuntu-19.04-disco
               components: CPython=3.7.3-64bit azstor.blob=2.0.1 azstor.file=2.0.1 crypt=2.8 req=2.22.0
       transfer direction: Azure -> Azure
                  workers: disk=0 xfer=48 md5=0 crypto=0
                 log file: None
                  dry run: False
              resume file: None
                  timeout: connect=10 read=200 max_retries=1000
              source mode: StorageModes.Auto
                dest mode: StorageModes.File
         server side copy: True
                  skip on: fs_match=False lmt_ge=False md5=False
                   delete: extraneous=False only=False
                overwrite: True
                recursive: True
            rename single: False
         strip components: 0
              access tier: None
============================================

Steps to Reproduce

1. Create a storage account called storage-account, with a blob container blob-container and a file share called file-share
2. Create a SAS token that isn't limited by IP or otherwise restricted on any scopes, store this as an env. var AZ_SAS_TOKEN
3. upload a simple text file containing the text "abcd" to the blob container
4. attempt to copy the contents of blob-container into file-share using the following command

blobxfer synccopy --remote-path https://storage-account.[snip]/blob-container?[sas] --sync-copy-dest-mode file --sync-copy-dest-storage-account storage-account --sync-copy-dest-storage-account $AZ_SAS_TOKEN --sync-copy-dest-remote-path file-share

Observe the attached error logs

Expected Results

The file testfile.txt is copied to file-share and contains abcd

Actual Results

testfile.txt isn't written to file-share

Additional Logs

2019-11-01 16:19:50.126 INFO blobxfer.operations.synccopy:_run:825 blobxfer start time: 2019-11-01 16:19:50.125955+10:30
2019-11-01 16:19:50.126 DEBUG blobxfer.operations.synccopy:_initialize_transfer_threads:271 spawning 48 transfer threads
2019-11-01 16:19:51.622 ERROR blobxfer.operations.synccopy:start:919 'content-length'
Traceback (most recent call last):
  File "***/.local/lib/python3.7/site-packages/blobxfer/operations/synccopy.py", line 917, in start
    self._run()
  File "***/.local/lib/python3.7/site-packages/blobxfer/operations/synccopy.py", line 834, in _run
    for src_ase, dst_ase in self._bind_sources_to_destination():
  File "***/.local/lib/python3.7/site-packages/blobxfer/operations/synccopy.py", line 775, in _bind_sources_to_destination
    self._general_options.dry_run):
  File "***/.local/lib/python3.7/site-packages/blobxfer/operations/azure/__init__.py", line 448, in files
    creds, options, dry_run):
  File "***/.local/lib/python3.7/site-packages/blobxfer/operations/azure/__init__.py", line 659, in _populate_from_list_blobs
    ase = self._populate_from_arbitrary_url(rpath)
  File "***/.local/lib/python3.7/site-packages/blobxfer/operations/azure/__init__.py", line 640, in _populate_from_arbitrary_url
    remote_path, int(response.headers['Content-Length']))
  File "***/.local/lib/python3.7/site-packages/requests/structures.py", line 52, in __getitem__
    return self._store[key.lower()][1]
KeyError: 'content-length'
Traceback (most recent call last):
  File "***/.local/bin/blobxfer", line 10, in <module>
    sys.exit(cli())
  File "/usr/lib/python3/dist-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3/dist-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/python3/dist-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3/dist-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/click/decorators.py", line 64, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "***/.local/lib/python3.7/site-packages/blobxfer_cli/cli.py", line 1125, in synccopy
    ctx.general_options, ctx.credentials, spec
  File "***/.local/lib/python3.7/site-packages/blobxfer/operations/synccopy.py", line 929, in start
    raise ex
  File "***/.local/lib/python3.7/site-packages/blobxfer/operations/synccopy.py", line 917, in start
    self._run()
  File "***/.local/lib/python3.7/site-packages/blobxfer/operations/synccopy.py", line 834, in _run
    for src_ase, dst_ase in self._bind_sources_to_destination():
  File "***/.local/lib/python3.7/site-packages/blobxfer/operations/synccopy.py", line 775, in _bind_sources_to_destination
    self._general_options.dry_run):
  File "***/.local/lib/python3.7/site-packages/blobxfer/operations/azure/__init__.py", line 448, in files
    creds, options, dry_run):
  File "***/.local/lib/python3.7/site-packages/blobxfer/operations/azure/__init__.py", line 659, in _populate_from_list_blobs
    ase = self._populate_from_arbitrary_url(rpath)
  File "***/.local/lib/python3.7/site-packages/blobxfer/operations/azure/__init__.py", line 640, in _populate_from_arbitrary_url
    remote_path, int(response.headers['Content-Length']))
  File "***/.local/lib/python3.7/site-packages/requests/structures.py", line 52, in __getitem__
    return self._store[key.lower()][1]
KeyError: 'content-length'

[alfpark]

I believe your command is malformed, try:

blobxfer synccopy --storage-account [sa1] --sas [blob-container-sas] --remote-path blob-container --sync-copy-dest-storage-account [sa2] --sync-copy-dest-sas [file-share-sas] --sync-copy-dest-remote-path file-share --sync-copy-dest-mode file --no-server-side-copy

A few notes:

1. You cannot use container/fileshare-level SAS tokens for cross-mode copies at the container/fileshare-level to the same storage account. You can either use the storage account key or account-level SAS tokens that give access to both services. I'll add a note in the limitations doc about this.
2. Blobxfer has direct env vars for things like storage accounts and sas tokens, see the CLI docs: https://github.com/Azure/blobxfer/blob/master/docs/10-cli-usage.md
3. While testing, I uncovered that --storage-url was not being accepted by synccopy. Additionally, --sync-copy-dest-storage-url was not properly detecting the dest mode from the URL. These bugs will be fixed.

[jburger]

Thanks for the fast response!

I tried your command, it appeared to fail differently:

2019-11-02 11:19:55.355 ERROR - Server side copy enabled with non-Block destination mode: StorageModes.File

So I dropped the explicit --sync-copy-dest-mode file argument to let it drop back to auto, and it while it doesn't raise an error, it also doesn't copy anything.

============================================
         Azure blobxfer parameters
============================================
         blobxfer version: 1.9.3
                 platform: Linux-5.0.0-32-generic-x86_64-with-Ubuntu-19.04-disco
               components: CPython=3.7.3-64bit azstor.blob=2.0.1 azstor.file=2.0.1 crypt=2.8 req=2.22.0
       transfer direction: Azure -> Azure
                  workers: disk=0 xfer=48 md5=0 crypto=0
                 log file: None
                  dry run: False
              resume file: None
                  timeout: connect=10 read=200 max_retries=1000
              source mode: StorageModes.Auto
                dest mode: StorageModes.Auto
         server side copy: True
                  skip on: fs_match=False lmt_ge=False md5=False
                   delete: extraneous=False only=False
                overwrite: True
                recursive: True
            rename single: False
         strip components: 0
              access tier: None
============================================
2019-11-02 11:35:40.740 INFO - blobxfer start time: 2019-11-02 11:35:40.739976+10:30
2019-11-02 11:35:40.740 DEBUG - spawning 48 transfer threads
2019-11-02 11:35:42.139 DEBUG - 1 remote files to sync, waiting for copy completion of approx. 0.0000 MiB
2019-11-02 11:35:43.430 INFO - elapsed copy time and throughput of 0.0000 GiB: 1.292 sec, 0.0001 Mbps (0.000 MiB/sec)
2019-11-02 11:35:43.431 INFO - blobxfer end time: 2019-11-02 11:35:43.431000+10:30 (elapsed: 2.691 sec)

Some more context around what I'm attempting to do:

• using a storage account level SAS token for source and destination
• copy the contents of a blob container to a fileshare in the same storage account
• see this comment for a description of the networking configuration nuances on these storage accounts CannotVerifyCopySource error - recursive copy from blob to file share on same storage account azure-storage-azcopy#721 (comment) I have a feeling it is related?

I uncovered that --storage-url was not being accepted by synccopy

Thanks - that could be a simpler option!

[alfpark]

Apologies, I forgot to include the important parameter:

--no-server-side-copy

By default, it will try to use server side copy, but that is not supported to an Azure File destination.

[alfpark]

Closing, please re-open if issue has not been solved.