Extracting morphological changes in nanocrystals using in situ liquid cell microscopy
Improving Automation of Frame Extraction
As we have discussed before, each one of our data sets contains thousands of video frames of STEM images from the nanocrystal growth experiments. However, due to the nature of scanning electron microscopy, only a small portion of these frames actually provide completely novel data. At the beginning of the project, we had to manually go through every frame of the videos and find which ones were completely refreshed and extract them. The ideal case would be to automate this process so that one could simply input the video and the program would extract a single complete frame for each scan of the region of interest.
Unfortunately, among all of the other issues we have run into, this is not a simple task and is not the highest priority. It is more a quality of life change rather than an issue that completely changes the functionality of the code. But we have managed to implement a method to greatly reduce the number of frames to be manually searched, from thousands to merely hundreds.
This routine searches for sharp discontinuity in the contrast of one line of pixels from the next. This works for most of the frames, as the raster line is often very apparent. However, for some of the frames the raster line is not so obvious and its detection is much more difficult. This means the detection routine will not remove all of the incomplete frames. But it will at least greatly reduce the number of frames to be manually searched from around 1500 to around 300.
Further enhancement of the routine would involve flagging the frames that are complete (or lack this sharp discontinuity) for extraction. However, there are often multiple frames that are completely refreshed and are essentially exactly the same. Therefore, the routine would need to flag only the first of each of these sets of images that are complete, on a periodic basis. For example, perhaps every 100 frames or so there are a set of 5 frames that are completely refreshed. The routine would need to flag the first of the five frames each time they roll around (every 100 frames) and extract those flagged frames. This is not something we anticipate finishing, and is left as an open opportunity. The current routine removes most of the incomplete frames so that manual searching is much faster.