Produces many 0 KB size files when "Re-Insert Available Exif Metadata" enabled

[fynnay]

Big thanks for the awesome guide and this tool ❤️

My specs:
Windows 10
Yes, my takeout consist of only the Google Photos
GooglePhotosExportOrganizer v1.1.1
Options: ✅ Use Extracted Location ✅ Keep Original Name 🔲 Re-Insert Availalbe EXIF Metadata

Description
When "Re-Insert Available Exif Metadata" option is enabled a lot of files get corrupted (aka become 0 KB in size). I haven't been able to find a specific pattern for which kind of files this happens to but it seems to only affect .jpg files. The app also crashes during the organize process - I never get to see the "Step 4: Tidy Up" page. Is there a log somewhere?

If i run organize without "Re-Insert Available Exif Metadata" it works well.

[Alamantus]

Hi @fynnay, thanks for the details!

It may take me a bit of time to look into it for you, but I'll let you know as soon as I find anything that might be causing this issue.

Quick question: does your Google Photos takeout also include a .json file for each of the image files in the folder? I haven't checked if Google has changed their takeout format since I first launched the app, so I'm wondering if there's some change in file structure that the code isn't anticipating...

As for logs, I do have a _DEBUG version for the current release that has the developer tools panel open by default. I'm not sure if it will log errors from the organize process there, but it might be worth checking out? If you end up running it in the debug version and see errors appear in the Console tab, please save the output by right-clicking inside the console tab and choose "Save As..." to save the .log file and upload it here for me. With any luck, we'll get some kind of useful error message! I'll need to add some kind of error logging in a future version...

[fynnay]

Hi, @Alamantus . Thanks a lot for taking the time to reply.

Yes, my Google Photos takeout includes .json files for each file.

📜 The .json file metadata files look like this:

{
  "title": "IMG_2539.JPG",
  "description": "",
  "imageViews": "13",
  "creationTime": {
    "timestamp": "1518684697",
    "formatted": "Feb 15, 2018, 8:51:37 AM UTC"
  },
  "photoTakenTime": {
    "timestamp": "1467056057",
    "formatted": "Jun 27, 2016, 7:34:17 PM UTC"
  },
  "geoData": {
    "latitude": 0.0,
    "longitude": 0.0,
    "altitude": 0.0,
    "latitudeSpan": 0.0,
    "longitudeSpan": 0.0
  },
  "geoDataExif": {
    "latitude": 0.0,
    "longitude": 0.0,
    "altitude": 0.0,
    "latitudeSpan": 0.0,
    "longitudeSpan": 0.0
  },
  "url": "https://lh3.googleusercontent.com/Qz8qyrw7-H3lWY-5gxTN_LwjwsL7uVT_EOGbj_I8_CTn7ulQqvS100P4OrjgyRObxatqiQ0NQTV_gPfsLFLIYwPDIYC34_vkBsEhRRM",
  "googlePhotosOrigin": {
    "mobileUpload": {
      "deviceType": "IOS_PHONE"
    }
  },
  "photoLastModifiedTime": {
    "timestamp": "1626157271",
    "formatted": "Jul 13, 2021, 6:21:11 AM UTC"
  }
}

Album information is saved within each Album sub-folder in a metadata.json file

📜 The Album info metadata.json files look like this:

{
  "title": "Album Name",
  "description": "",
  "access": "protected",
  "date": {
    "timestamp": "1579043378",
    "formatted": "Jan 14, 2020, 11:09:38 PM UTC"
  },
  "location": "",
  "geoData": {
    "latitude": 0.0,
    "longitude": 0.0,
    "altitude": 0.0,
    "latitudeSpan": 0.0,
    "longitudeSpan": 0.0
  }
}

I've tried using the _DEBUG for Windows but unfortunately no useful messages seem to have appeared. However I found that only .jpg files - though not all of them - seem to be affected. All other files work fine.

Additionally I tried to make a small test folder with an image that previously failed and one that succeeded along with their .json files - which appear to have the same syntax. Curiously running the ExportOrganizer on this folder moved both files perfectly fine.

Other observations:

• When processing a large folder the window is Not Responding (the window title says so, too). The progress bar is moving anyways
• Before the program crashes (which it always does when processing my large folder) the progress bar fills up and a checkmark is shown
• No log messages are written to the console during the process (neither in Verbose mode)
• These messages appear when the program is first launched:

🖼️ Screenshot of log

[Alamantus]

@fynnay Thanks for your detailed reply! Your observation that it only is a problem for .jpg files makes sense—only JPEG format files can use Exif data.

It's interesting that separating some files that previously failed actually ran successfully. I think the main problem with my app is that it tries to just grab all the file references and run them asynchronously at the same time. When I initially built the app, my Google takeout was much smaller than many of the people I've seen use the app since, which probably made this not an issue, but with larger takeout collections, I could imagine this process would be an issue. In a future version I'd like to change the both the extract and organize processes to use queues rather than trying to handle everything whenever it's able, which I believe may ultimately solve this issue at the cost of it potentially taking a little longer.

Anyway, since separating the files works, I'd recommend for now that you just split the folders into smaller sets of folders to run the Organize process on, if possible, and just point the output folder to the same place so it all merges together. If that works, I may update the instructions in the meantime while I wait to have time to work on the queue improvement.

[fynnay]

Ah yes that would make sense. Maybe some of the "threads" are crashing because of a lack of Memory/CPU power/Disk IO while processing many images.

[alistairbanks]

Just to add weight to this, i see the same 0Kb files error on huge Google Photos takeout of 17 * 10Gb .zip files on Win11 Pro, Surface Pro 8 i7 with lots of resource.

[nrschoen]

I also have this issue and I figured out that it doesn't work for pictures that have been manually uploaded from my android. So if I received a picture over whatsapp for instance and then added it to google photos then it shows up as a 0 KB file in the final folder :(

[petebaker17]

I have found that although this issue happened when I tried it on the full set of photos, if I select one sub-folder at a time (e.g. 2020) photos, then this issue doesn't happen. So it must be due to the volume of photos to process. Hopefully this is helpful if anyone is reading this