Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FIX: hanging stream identification #189 #190

Closed
wants to merge 3 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions fido/fido.py
Original file line number Diff line number Diff line change
Expand Up @@ -532,10 +532,11 @@ def blocking_read(self, file, bytes_to_read):
buffer = b''
while bytes_read < bytes_to_read:
readbuffer = file.read(bytes_to_read - bytes_read)
last_read_len = len(readbuffer)
buffer += readbuffer
bytes_read = len(buffer)
# break out if EOF is reached.
if readbuffer == '':
bytes_read += last_read_len
# break out if EOF is reached, that is zero bytes read.
if last_read_len < 1:
break
return buffer

Expand Down
37 changes: 37 additions & 0 deletions tests/test_fido.py
Original file line number Diff line number Diff line change
@@ -1,13 +1,50 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from __future__ import print_function

import io
import tempfile
from time import sleep

from fido import fido
from fido.fido import PerfTimer

# Magic number for fmt/1000.
MAGIC = b"\x5A\x58\x54\x61\x70\x65\x21\x1A\x01"

def test_perf_timer():
timer = PerfTimer()
sleep(3.6)
duration = timer.duration()
assert duration > 0

def test_file_identification():
"""Reference for Fido-based format identification
1. Create a byte-stream with a known magic number and serialise to tempfile.
2. Call identify_file(...) to identify the file against Fido's known formats.
"""
# Create a temporary file on the host operating system.
tmp = tempfile.mkstemp()
tmp_file = tmp[1]

# Write to the file our known magic-number.
with open(tmp_file, "wb") as new_file:
new_file.write(MAGIC)

# Create a Fido instance and call identify_file. The identify_file function
# will create and manage a file for itself.
f = fido.Fido()
f.identify_file(tmp_file)
Comment on lines +21 to +37
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice tests, Carl. I think this one could be a good use case for the tmp_path fixture provided by pytest. It gives us a pathlib.Path object which has a nice API. pytest also takes care of removing old temporary directories, always keeping the last three.

Suggested change
def test_file_identification():
"""Reference for Fido-based format identification
1. Create a byte-stream with a known magic number and serialise to tempfile.
2. Call identify_file(...) to identify the file against Fido's known formats.
"""
# Create a temporary file on the host operating system.
tmp = tempfile.mkstemp()
tmp_file = tmp[1]
# Write to the file our known magic-number.
with open(tmp_file, "wb") as new_file:
new_file.write(MAGIC)
# Create a Fido instance and call identify_file. The identify_file function
# will create and manage a file for itself.
f = fido.Fido()
f.identify_file(tmp_file)
def test_file_identification(tmp_path):
"""Reference for Fido-based format identification
1. Create a byte-stream with a known magic number and serialise to tempfile.
2. Call identify_file(...) to identify the file against Fido's known formats.
"""
# Create a temporary file on the host operating system.
tmp_file = tmp_path / "file"
# Write to the file our known magic-number.
tmp_file.write_bytes(MAGIC)
# Create a Fido instance and call identify_file. The identify_file function
# will create and manage a file for itself.
f = fido.Fido()
f.identify_file(str(tmp_file))

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So the original ideas here came from @ross-spencer (that's intended to give credit not share blame for temp file management). I agree that these could be better for test purposes. The PR will remain draft for now as I'm going to test a couple of ideas around the API.


def test_stream_identification():
"""Reference for Fido-based format identification
1. Create a byte-stream with a known magic number.
2. Call identify_stream(...) to identify the file against Fido's known formats.
"""
# Create the stream object with the known magic-number.
fstream = io.BytesIO(MAGIC)
# Create a Fido instance and call identify_stream. The identify_stream function
# will work on the stream as-is. This could be an open file handle that the
# caller is managing for itself.
f = fido.Fido()
f.identify_stream(fstream, "filename to display", extension=False)