Temporary CDXJ files are not found by replay

[machawk1]

Using latest master 63b2805, Python 3.7.7 macOS 10.15.5

ipwb index samples/warcs/5mementos.warc | ipwb replay
Processing WARC records in 5mementos.warc complete
IPWB replay started on http://localhost:5000
 * Serving Flask app "ipwb.replay" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off

(access http://localhost:5000/ in a browser)

Retrieving URI-Ms from /var/folders/k_/0fg5752j3vl3b814h2qyml_00000gn/T/tmpaai_dd3d.cdxj
[Errno 2] No such file or directory: '/var/folders/k_/0fg5752j3vl3b814h2qyml_00000gn/T/tmpaai_dd3d.cdxj'
(<class 'FileNotFoundError'>, FileNotFoundError(2, 'No such file or directory'), <traceback object at 0x10a098c80>)
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1949, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1935, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/local/lib/python3.7/site-packages/ipwb/replay.py", line 579, in showLandingPage
    mementoInfo = calculateMementoInfoInIndex(iFile)
  File "/usr/local/lib/python3.7/site-packages/ipwb/replay.py", line 931, in calculateMementoInfoInIndex
    indexFileContents = get_web_archive_index(cdxjFilePath)
  File "/usr/local/lib/python3.7/site-packages/ipwb/backends.py", line 92, in get_web_archive_index
    response = fetch_local_index(path)
  File "/usr/local/lib/python3.7/site-packages/ipwb/backends.py", line 64, in fetch_local_index
    with open(path, 'r') as f:

This might be due to @ibnesayeed adaptation of the temp file to use the native library instead of our own temporary file generation approach. Perhaps the file is not persistent.

[machawk1]

This also shows a need for a piping test to evaluate the integration "chaining" of the indexer to the replay code.

[machawk1]

b6bcc97 in issue-696 attempts to call the ipwb command, which does appear to be available in the pytest code. We want to test the command and not just the code itself here. Is this possible from within pytest? Can we have a requisite install of the module just for the test?

[machawk1]

As an alternative, args could also be passed to main's checkArgs_replay() but that does get to the crux of the replication. It should be done for unit testing anyway.

[ibnesayeed]

I think I know the issue and how to fix it. We should be using tempfile.mkstemp() instead of tempfile.TemporaryFile().

[machawk1]

Yes, probably:

Unlike TemporaryFile(), the user of mkstemp() is responsible for deleting the temporary file when done with it.