-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Bug] run_one_image demo creates bad resutls #5
Comments
It seems that the problem is caused by the area value of trimap. Please ensure that the values of trimap in the foreground, unknown, and background areas are 255, 128, and 0 respectively. |
same issue. Have you solved this issue?? In my case, even though I double-checked the trimap value and corrected wrong value to one of (0, 128, 255), I could not get a result. Could you share the correct trimap? Above both images are provided by DiffMate but they do not work. |
@jseobyun Hi, can you share me examples to run the file run_one_image.py? I couldn't run this file |
I cannot reproduce this bug in my recommending environment, even though I do not think this is caused by unpair envs. Maybe using the fixed command of run_one_image.py demo can help. |
I am facing the same issue |
@YihanHu-2022 the values of trimap must be 0, 0.5 and 1 according to your code (for example). If I replace them on 0, 128 and 255 as you proposed above, then I will get I also tried running |
@YihanHu-2022 I tried both ViTB and ViTS_1024, both produce bad result.
Here is my environment in docker container:
@YihanHu-2022 can you pack your environment into a docker image so that we can see why it fails? |
same issue with you guys |
The trimap is not the cause. I added three lines below to guarantee only three values (0/0.5/1) from trimap. Same result def get_data(image_dir, trimap_dir):
"""
Get the data of one image.
Input:
image_dir: the directory of the image
trimap_dir: the directory of the trimap
"""
image = Image.open(image_dir).convert('RGB')
image = F.to_tensor(image).unsqueeze(0)
trimap = Image.open(trimap_dir).convert('L')
trimap = F.to_tensor(trimap).unsqueeze(0)
# force tri-values in trimap
trimap[trimap > 0.9] = 1.00000
trimap[(trimap >= 0.1) & (trimap <= 0.9)] = 0.50000
trimap[trimap < 0.1] = 0.00000
return {
'image': image,
'trimap': trimap
} |
The problem is in saving output. Use this fixed function instead def infer_one_image(model, input, save_dir=None):
output = model(input)
# output = F.to_pil_image(output).convert('RGB')
# output.save(opj(save_dir))
output = cv2.cvtColor(output, cv2.COLOR_GRAY2RGB)
cv2.imwrite(opj(save_dir), output)
return None |
@ReddyNick yes, you saved the world ! I created a PR here #8 From my test, Diffmatte is faster than AEMatte although they both produce good result |
@ReddyNick @wangjia184 @YihanHu-2022 Am I the only one here who is unable to run DiffMatte ViTS-1024 on image of resolution 1920 x 2880 without facing CUDA out of memory error?) On image of resolution 1920 x 1280 everything is ok. It seems to me that gradient calculation is not disabled somewhere. I tried using |
Hi, this is a valuable question and is indeed what our team currently works on. The huge cuda memory need is caused by the global attention employed in the ViT backbone. To address this problem you can refer to the solution at hustvl/ViTMatte#10. BTW we will share another work that provides a better method to handle high-res matting with the mature ViT series backbone network, and I'll share the link under this issue. |
@YihanHu-2022 First of all, thanks for reacting to the issues. Here's an example code: from diffmatte_run import infer_one_image as infer_diffmate, init_model as init_model_diffmatte
app = Flask(__name__)
@app.route("/", methods=["POST"])
def run_inference():
# ... code to process the image
try:
image_alpha = infer_diffmate(
init_model_diffmatte(args.model, args.checkpoint, args.device, "ddim10"),
{
"image": F.to_tensor(image_rgb).unsqueeze(0),
"trimap": F.to_tensor(image_trimap).unsqueeze(0),
},
)
output_format = "png"
byte_arr = io.BytesIO()
image_alpha.save(byte_arr, format=output_format)
byte_arr.seek(0)
return Response(byte_arr.getvalue(), mimetype=f"image/{output_format}")
except Exception as e:
return jsonify({"error": str(e), "traceback": traceback.format_exc()}), 500
finally:
if args.device == "gpu":
torch.cuda.empty_cache()
if __name__ == "__main__":
app.run(host='0.0.0.0', port=8092, debug=True, use_reloader=True) If you are interested, I can you provide full code of the Flask app. |
@LukaGiorgadze Start a subprocess for every inferrence, that is how I use it:) |
As this issue seems to be addressed, I gonna close it up. |
I followed the set up, the only differences are
I got the following results using ViTS_1024 and associated weights
There is no warnings/errors during the inference.
Please advise
The text was updated successfully, but these errors were encountered: