Posts Tagged ‘fs’
install vs. cp; and mmap
By chys on May 8th, 2009If we hand write a Makefile, we should always stick to install instead of using cp for the installation commands. Not only is it more convenient, but it does things right (cp does things wrong).
For example, if we attempt to update /bin/bash, which is currently running, with “cp ... /bin/bash”, we get a “text busy” error. If we attempt to update /lib/libc.so.6 with “cp ... /lib/libc.so.6”, then we either get “text busy” (in ancient versions of Linux) or breaks each and every running program within a fraction of a second (in recent versions of Linux). install does the thing right in both situations.
The reason why cp fails is that it simply attempts to open the destination file in write-only mode and write the new contents. This causes problem because Linux (and all contemporary Unices as well as Microsoft Windows) uses memory mapping (mmap) to load executables and dynamic libraries.
The contents of an executable or dynamic library are mmap’d into the linear address space of relevant processes. Therefore, any change in the underlying file affects the mmap’d memory regions and can potentially break programs. (MAP_PRIVATE guarantees changes by processes to those memory regions are handled by COW without affecting the underlying file. On the contrary, POSIX leaves to implementations whether COW should be used if the underlying file is modified. In fact, for purpose of efficiency, in Linux, such modifications are visible to processes even though MAP_PRIVATE may have be used.)
There is an option MAP_DENWRITE which disallows any modification to the underlying file, designed to avoid situations described above. Executables and dynamic libraries are all mmap’d with this option. Unfortunately, it turned out MAP_DENYWRITE became a source of DoS attacks, forcing Linux to ignore this option in recent versions.
Executables are mmap’d by the kernel (in the execve syscall). For kernel codes, MAP_DENYWRITE still works, and therefore we get “text busy” errors if we attempt to modify the executable.
On the other hand, dynamic libraries are mmap’d by userspace codes (for example, by loaders like /lib/ld-linux.so). These codes still pass MAP_DENYWRITE to the kernel, but newer kernels silently ignores this option. The bad consequence is that you can break the whole system if you think you’re only upgrading the C runtime library.
Then, how does install solve this problem? Very simple – unlinking the file before writing the new one. Then the old file (no longer present in directory entries but still in disk until the last program referring to it exits) and the new file have different inodes. Programs started before the upgrading (continuing using the old file) and those after the upgrading (using the new version) will both be happy.
Ext4 data loss
By chys on March 19th, 2009I, too, have experienced data losses in ext4 partitions.
There was some problem with my X that hangs the system about once a week (I was upgrading my X system and drivers too aggressively) so sometimes I had to hard reboot my computer (even magic SysRq does not respond in such cases). I lost most or all of my KDE settings after the reboots, and for one time also all my Thunderbird settings. This never happened until I migrated to ext4..
So I converted all my ext4′s back to ext3 and downgraded the kernel to 2.6.27 which is considered by Gentoo as stable. Loss of KDE settings is no big deal, but I really don’t want to have a kernel bug erase my codes, or homework that is due tomorrow, or diary I’ve kept for eight years.
Anyway, the performance of ext4 is really good..
References – Other recent reports of ext4 data losses:
[1] https://bugs.edge.launchpad.net/ubuntu/+source/linux/+bug/317781/comments/45
[2] http://www.h-online.com/open/Ext4-data-loss-explanations-and-workarounds–/news/112892
[3] http://www.h-online.com/open/Possible-data-loss-in-Ext4–/news/112821
[4] http://cookinglinux.cn/ext4-lose-data.html (Chinese)
Migrating to EXT4
By chys on December 30th, 2008Ext4, the successor to ext3 which was formerly known as ext4dev, is marked stable in Linux kernel 2.6.28, meaning the Linux kernel team now recommends using ext4 in production.
To convert a file system from ext3 to ext4, use
tune2fs -O extents /dev/DEV
and remount the file system as ext4. (Two e2fsck runs are recommended before and after tune2fs.) Some documentations also include the -E test_fs option. This is not necessary now since ext4 is no longer experimental.
Finally do not forget to modify /etc/fstab.
An ext4 file system created this way is not a “true” ext4 – the extents feature, the main advantage of ext4 comapred to ext3, is not automatically applied to old files. New files created afterwards are in the extents format.
Unlike the 100% backward compatibility of ext3 with ext2, an ext4 file system can no longer be mounted as if it were an ext3, unless the extents feature is disabled. (If you want to disable extents, why not simply use ext3?)
This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.
Shrink an LVM partition
By chys on October 10th, 2008Gentoo’s Documentation says it is easier to increase the size of an LVM partition than to shrink it, so they recommend starting with smaller partitions and then increase them as needed.
Of course they are not saying shrinking is impossible, but only less easy. The most important thing is the FS must be reduced BEFORE the volume, or the FS will get damaged.
Example: Shrink partiton /dev/vg/home with ext3 filesystem from 30G to 12G:
e2fsck /dev/vg/home (resize2fs requires a check before resizing)
resize2fs /dev/vg/home 10G (resize FS to 10G)
lvreduce -L12G /dev/vg/home (reduce partition)
resize2fs /dev/vg/home (extend FS to the whole partition)
Someone (I’m not sure who..) recommended doing two resizes like this claiming it is safer.
This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.
