May 2020 to August 2020
Name: Ajay Bharadwaj
Organization: CRIU
Mentors: Andrei Vagin, Dmitry Safonov
Official Documentation: Validating files on restore
While checkpointing, CRIU stores some meta-information about the file and this is validated while restoring to make sure the right file is being restored (This is especially necessary for ELF files since there is a risk of restoring executables or libraries of a different version). Previously, the only meta-information stored was the file size, which meant that the validation wasn't very strong. The goal of this project was to extend this meta-information to include the build-ID of the file (ELF files only) and the CRC32C checksum of the file. Currently, the file size check is used only as a preliminary test, after which, either the build-ID or the checksum method is used.
The build-ID is a "strongly unique embedded identifier" that (If present) is stored in a particular note section of ELF files. The build-ID method is much less intensive compared to the checksum method while still being a much stronger check than simply comparing the file size and therefore CRIU uses this method wherever possible by default. The checksum method is used a fallback when the build-ID cannot be obtained.
CRC32C checksum is a slight variation of the better known CRC32C checksum. The only difference is the polynomial being used. CRC32C uses the Castagnoli polynomial (0x82F63B78 in little-endian notation) and CRC32 uses 0xEDB88320 (In little-endian notation). CRC32C is used since it is supported by the hardware in newer Intel CPUs. The checksum is calculated on the required bytes depending on the configuration set - Entire file, First N bytes of the file, or Every Nth byte of the file. Here, N is the checksum parameter.
To explicitly use the checksum method for the entire file:
criu ... --file-validation checksum-full
To explicitly use the checksum method for the first 2048 bytes of the file:
criu ... --file-validation checksum --checksum-parameter 2048
To explicitly use the checksum method for every 2048th byte of the file:
criu ... --file-validation checksum-period --checksum-parameter 2048
To explicitly use only the filesize method:
criu ... --file-validation filesize
This project also modified the ZDTM testing framework to support the flag 'crfail-restore' which is used to indicate that the test is expected to fail during restore. A total of 10 automated tests have been added to test the new features.
| Commit Number | Commit Name |
|---|---|
| Commit 1/10 | Add additional fields to RegFileEntry |
| Commit 2/10 | Add build-ID validation functionality |
| Commit 3/10 | Add CLI option for build-ID |
| Commit 4/10 | First fixup commit for build-ID validation |
| Commit 5/10 | Second fixup commit for build-ID validation |
| Commit 6/10 | Add CLI option for checksum |
| Commit 7/10 | Add checksum validation functionality |
| Commit 8/10 | Add crfail-restore flag to ZDTM |
| Commit 9/10 | Add build-ID tests to ZDTM |
| Commit 10/10 | Add checksum tests to ZDTM |
The values shown below are the average times it took to finish the ZDTM tests over multiple runs, and are only to indicate the impact each method has in general.
Each test has 3 files and each test is run 3 times (In Host, Namespace and User Namespace). For each file the checksum/build-ID is obtained twice (During dump and restore) therefore the function to find checksum/build-ID is called 18 times overall per test.
For reference, these tests were run on tmpfs (To remove any disk latency) and on an undervolted i5 8300H.
zdtm/transition/shmem:
3 files of size: 0.09 MB, 2 MB and 0.17 MB approximately
| File Size | 3.782s |
| Build-ID | 4.153s (~9% increase) |
| Checksum (First 1024 bytes) | 4.465s (~18% increase) |
| Checksum (Entire File) | 4.722s (~24% increase) |
| Checksum (Every 1024th byte) | 4.498s (~19% increase) |
zdtm/static/maps04:
3 files of size: 0.1 MB, 2 MB and 0.17 MB approximately
| File Size | 35.317s |
| Build-ID | 35.720s (~1% increase) |
| Checksum (First 1024 bytes) | 35.919s (~2% increase) |
| Checksum (Entire File) | 36.679s (~4% increase) |
| Checksum (Every 1024th byte) | 36.476s (~3% increase) |
Calculating the CRC32C checksum can be made faster by using a lookup table.
I am immensely grateful to my mentors Andrei and Dmitry and the rest of the community as well for their guidance and feedback throughout the project. It was a wonderful oppurtunity to work with the CRIU community and I had a great time working on this project. I am extremely enthusiastic about continuing to contribute to CRIU.