Fixed an issue where empty rows would not be removed from manifests, and add test for load_df util function

Fixed an issue where empty rows would not be removed from manifests, and add test for load_df util function #3788