convert to unix line terminators

1 files changed, 364 insertions, 364 deletions

diff --git a/README b/README
index 28db56d..13f3c74 100644
--- a/README
+++ b/README
@@ -1,364 +1,364 @@
-README for ext4magic V-0.2.0

-

-

-You accidentally deleted files ?

-=================================

- 

-Now, you can try it with ext4magic - probably you will find many - but not all

-deleted files. ext4magic will not change the data on your partition.

-It write copies of found deleted files to a directory on a different file system.

-For that you need enough disk space on a ext4 or ext3 Linux file system. 

-

-This tool requires a working file system. If the partition table ore the file system

-damaged, ext4magic can not help. Then you should use a different recover tool.

-In addition to the recovery functions a lot of other functions are included.

-These functions allow a deep look into the file system and can also help to find 

-data and files which are not automatically recover.

-

-

-

-How does this work ?

-=====================

-

-A file in an ext3/4 filesystem consists of several parts. The name of the file

-and a Inode nummer are in data blocks of the directory. This Inode nummer is 

-a serial number for a data structure in a tabel of these structures. 

-These structures are called Inode and are the most important part of the file.

-In the Inode are included all properties of the file and the reference to 

-there data blocks. In the data blocks store the data of the file. In example, 

-all the bytes for a jpg image

-

-During the deletion of a file, be completely destroyed all refer to the data

-blocks in inode data. The content of data blocks are not destroyed, but the 

-block now marked as free.

-If you write new files, this free data blocks can reused for new files.

-The old inode is also marked as free and is also ready for reuse.

-Name and Inode number in the directory block are only marked deleted, 

-they are skipped for now when searching for file names in this directory.

-

-Deleted files can not re-assembled, the Inode data are unsuitable for 

-this purpose.  Exactly what the developers say.

-

-But there is the filesystem Journal. Journaling ensures the integrity of the 

-filesystem by keeping a log of the ongoing disk changes.

-After deleting a file, there you found a copy of the data block in which the 

-deleted Inode is included. Well, this copy is not usable for a recover. 

-The Inode is deleted, but perhaps there is also still an even older copy of 

-the same data block.

-If you find such an older block in the Journal, then you can find there the old

-intact Inode copy of the deleted file. And with such an old Inode, you can now

-undelete the file. You find in the Inode the properties and all refer to the 

-data blocks. In the directory you find the old file name.  With a little luck,

-the data blocks are not reused.

-

-This is the principle of ext4magic to recover from inode copies. 

-

-In the Journal there are not only inode copies.  You will also find tables with

-the block and inode allocation. This data are used in the magic functions

-for controlling the file carving. The functions of the file carving matched

-exactly to the respective properties of the file system types and these functions

-included into a multi-stage recover process. This feature is new in this version

-and currently only usable for ext3.

-

-

-

-----------------------------------------------------------------------------------

-

-

-How you can use ext4magic ?

-===========================

-

-You need, of course, the file system from which you want try to recover deleted 

-files. The safest way is to create an image of the partition. 

-Important, for this, the filesystem must umounted or readony mounted.

-

-For example: the filesystem is on /dev/sda1

-

- # dd  if=/dev/sda1  of=/path/to/image_name  bs=4096

-

-

-With the shell, you change to a directory, where enough free space to write 

-the data recovers.  You need also some options, but that later. 

-

-You can use ext4magic:

-

- # ext4magic  /path/to/image_name  options

-

-

-

-

-Not enough free space for a imagefile of the entire filesytem ?

--------------------------------------------------------------

-

-If you can use ext4magic from a rescue system or from a LINUX Live-system,

-or on a other LINUX system, do not mount the partition with the deleted files and use it directly 

-

-  # ext4magic /dev/sda1 options

-

-

-

-

-You can not restart the computer or umount the partition ?

----------------------------------------------------------

-

-Attempts to mount the partition readonly. The best way try to "umount" and then 

-"mount -o ro /dev/sda1" . If this ist noch posible?  try the following:

-

- # mount -o remount,ro,noload  /dev/sda1

-

-if the partition is now mounted readonly, use also

- # ext4magic /dev/sda1 options

-

-

-

-

-It is impossible to mount readonly ? 

-------------------------------------

-

-ext4magic still has a solution, but highly experimental. Please use only in 

-exceptional cases. Never use ext4magic for a not readonly mounted partition. 

-ext4magic read over the filesystem buffer from journal but the kernel write 

-unbuffered to journal.

-

-The first read of the journal is often ok, but all subsequent reads can read

-wrong data blocks from journal. So long the journal file is buffered, you read 

-wrong data blocks at the moment of the first read. The file system is operating

-normally without errors, but ext4magic reads wrong blocks from the Journal.   

-

-

-Workaround : ext4magic supports external journal. 

-You can create a copy of the filesystem journal with the "debug2fs" command. 

-Use this copy as external Journal for the mounted file system.

-But, if mounted readwrite, here also only the first backup will work good,

-after read the journal by debug2fs, the journal is also buffered and the next

-read by debug2fs results also a bad journal copy.

-

-

-  # debug2fs  -R  "dump <8> /path/to/journalbackup" /dev/sda1

-

-you can use this copy of Journal  

-

-  # ext4magic /dev/sda1 -j  /path/to/journalbackup options 

-

-ext4magic then only read journal data from this journal backup. 

-  

-Warning: This procedure is tested, it works, but please be very careful 

-with this feature. Remember, for ext4magic the file system is frozen at the time at

-which the journal copy created. Any subsequent changes will not recognize by ext4magic.

-This works only for a limited time if you continue to write into the file system.

-

-

-

-

---------------------------------------------------------------------------------------------------

-

-

-A few words about the new magic functions (current version 0.2.0)

- ============================================================

- These functions are designed to make undo of recursive deletes. It is a  multi-level recover

- and also restore files if no old journal copies can be found for this file.

-

-

-   1. recover files of the file system tree with the help of old inode copies.

-   2. recover all other inode copies which were not found in first stage. 

-   3. (currently only ext3) recover the remaining data blocks, using a file carving function (we say magic function)  

- 

-

-

- After an accidental deletion:  prevent all writing into this file system and if possible also

- prevent reading of this file system. Also reading overwrites old journal data

- which are needed for the restore. 

-

- Umount the file system, and use ext4magic before you mount the file system again,

- or create a copy of the file system and use this for the recover.

- Perform no file system check on this file system before.

-

-

- The magic functions are very user friendly because very few command options are required.

-

- Extensive testing has confirmed that magic-scan-functions are now stable with libmagic of file-5.04.

- Good support exists for: all text file types, a lot of image formats, 

- often-used video and audio file types, Open Office documents,

- PDF, RAR, TAR, CPIO, BZ2, ZIP, GZIP, 7Z ...

- 

- Many other file types are also found and restored with default function, but without examining

- the contents of the files. This works more or less. 

-

- Problems still exist with some multimedia formats and some documents. Not every file type

- can be restored only based on head and foot patterns. Some types of multimedia streams, splited or

- truncated files are hard to recover.

- The recovery of CD/DVD images and other file system containers is also problematic. This can only work 

- in file systems with 4KB block size.

- Sparse files, and very large files if not deleted in one step, can not be restored with this 

- function. (Bug:#017607)

- Of course, you can only find files when the "file" command recognize this file type. It is theoretically

- possible to enable the restore of unknown file types by an entry in the configuration file to "magic".

- 

-

- Some files are one (or few) byte too short. These are final zero byte.

- Most of these files can be repaired by appending zeros.

- The following command illustrates how attach two zero byte to a file.

-

- #echo -en "\0\0" >> file

-

-

- Some files are one or more bytes to long. These are often zero byte at the end of the restored file.

- You can see this at the end of a file. "hexdump-C file | tail "

- These files can be opened usually normal, possibly with a warning. Only a few programs block the

- processing of such files.  Here is an example,  how this can be fixed (xz compressed file)

-

-# ls -l test.xz

--rw-r--r-- 1 rob users 1005  4. Dez 12:54 test.xz

-# xz -t test.xz

-xz: test.xz: Compressed data is corrupt

-# xz -d test.xz

-xz: test.xz: Compressed data is corrupt

-# dd if=test.xz of=test_.xz bs=1 count=1004

-1004+0 Datensätze ein

-1004+0 Datensätze aus

-1004 Bytes (1,0 kB) kopiert, 0,0164605 s, 61,0 kB/s

-# xz -t test_.xz

-# xz -d test_.xz

-

- 

- The magic functions do not work particularly fast, but very efficient and can find some files

- that other tools can not recover. It also find very long files when the data are fragmented in the

- file system. Others file carving tools find here often no complete files, or recover data trash.

- Because of the previously running recover stages, the hit rate of this function is often very good.

- But, at very high fragmentation the chances are low for a successful recovery for many files.

- 

- In real file systems the magic function find also unfortunately some very old files.

- The idea, to prevent this by using the metadata from the journal, is definitely good, but, 

- in a real file system it works only limited. In test file systems it works very well, but in a real

- file system journal you find not always enough of these metadata to prevent the recover of very old files.

-

- 

-

---------------------------------------------------------------------

-

-

-

-Instructions to experimenting with new features, the magic functions.

----------------------------------------------------------------------

-

-Use no file system specially created for this purpose.

-Why?

- If you create a test file system, it is likely that all inode copies are included

- in the Journal. The first stage can restore all files, and you'll never see the

- magic functions in the third stage.

-

-

-Better is the following:

- Use an existing ext3 filesystem. The last hours should no run a global "find" or a backup tool

- in this file system. That too would write to many inode copies and to be easy to recover.

- umount this file system, and create a 1-to-1 copy of the file system.

- Now mount the file system copy and delete all or many files. Then umount the file system copy.

-

-

-

-Now you can test ext4magic with the deleted copy.

- You need free space for writing the recovered files.

- Assuming, the copy is "/dev/sdb1" and you have enough free

- space at "/home/test/"

-

- # ext4magic /dev/sdb1 -d /home/test/RECOVER -M

- if you have deleted all files.

-

- or

- # ext4magic /dev/sdb1 -d /home/test/RECOVER -m

- if not all files were deleted.

-

-

- It will automatically search for the time of the last deletion.

- And with a little delay should start the recover. You can now only wait. Depending on the

- number of deleted files can take a long time. Then you can compare the files with 

- the original file system.

-

-

---------------------------------------------------------------------------------------------

-

-The Options of ext4magic

-=========================

-

-ext4magic has a lot of options, here are just a small overview.

-Detailed information take from the manpage.

-

-One option must always be specified, the file system.

-

-

-Information Options  -S -J -H -T 

----------------------------------

-This can display information from the file system, the journal ore the transactions from journal.

-

-

-

-Selections  -I -B  -f

------------------------

-select the specific inode, blocks or file names for the information- and action options.

-

-

-

-Time Options -a -b -t

-------------------------

-These are important control options. This indicates the time window for searching for files.

-

-

-

-File input and output options  -d -i -j

----------------------------------------

-This can be specified, the output directory, a input file list and an external journal file

-

-

-

-Action Options  -l -L -r -R -m -M

-----------------------------------

-For select of the various listing- and recover actions.

-

-

-

-

----------------------------------------------------------------------------------------------

-

-

-Some common problems

-

-Command not found 

-------------------

-ext4magic is installed to /usr/local/sbin/ 

-This directory is only included in the PATH if you use root as a login shell.

-For a full root environment  use "su -l" for the user change.

-

-

-

-Manpage not found

------------------

-The manpage is installed under /usr/local/*/man/man8/

-Check if the MANPATH variable include the following directories.

-

- /usr/local/man /usr/local/share/man

-

-

-

-ext4magic nothing works

------------------------

-two possible causes: 

-- either you are not root

-- or the time options are not set correctly. Only the magical functions automatically search 

-  for the best time window, all other options use default time values. (See manpage) 

-

-

-

-----------------------------------------------------------------------------------------------

-

-Known Bugs

-

-Only on big endian environments, there are some incorrect outputs of time stamps, and missing of

-deleted directory entries. (BUG #017304 #017305) 

-These errors occur only if the journal is not read and so only called functions of libext2fs for

-printout of inode and directorys. All journal options and the file restoring are not affected. 

-The error is not within ext4magic and can not be compensated in ext4magic. This would be patched

-in libext2fs. The error is very rare and not significant. If anyone needs a patch for this,

-no problem, within ext4magic the problem is solved. It is also possible to write an unofficial patch

-for libext2fs. I just think that nobody will really need it. Otherwise, send a request to the ext4magic 

-mailing list.

-

+README for ext4magic V-0.2.0
+
+
+You accidentally deleted files ?
+=================================
+ 
+Now, you can try it with ext4magic - probably you will find many - but not all
+deleted files. ext4magic will not change the data on your partition.
+It write copies of found deleted files to a directory on a different file system.
+For that you need enough disk space on a ext4 or ext3 Linux file system. 
+
+This tool requires a working file system. If the partition table ore the file system
+damaged, ext4magic can not help. Then you should use a different recover tool.
+In addition to the recovery functions a lot of other functions are included.
+These functions allow a deep look into the file system and can also help to find 
+data and files which are not automatically recover.
+
+
+
+How does this work ?
+=====================
+
+A file in an ext3/4 filesystem consists of several parts. The name of the file
+and a Inode nummer are in data blocks of the directory. This Inode nummer is 
+a serial number for a data structure in a tabel of these structures. 
+These structures are called Inode and are the most important part of the file.
+In the Inode are included all properties of the file and the reference to 
+there data blocks. In the data blocks store the data of the file. In example, 
+all the bytes for a jpg image
+
+During the deletion of a file, be completely destroyed all refer to the data
+blocks in inode data. The content of data blocks are not destroyed, but the 
+block now marked as free.
+If you write new files, this free data blocks can reused for new files.
+The old inode is also marked as free and is also ready for reuse.
+Name and Inode number in the directory block are only marked deleted, 
+they are skipped for now when searching for file names in this directory.
+
+Deleted files can not re-assembled, the Inode data are unsuitable for 
+this purpose.  Exactly what the developers say.
+
+But there is the filesystem Journal. Journaling ensures the integrity of the 
+filesystem by keeping a log of the ongoing disk changes.
+After deleting a file, there you found a copy of the data block in which the 
+deleted Inode is included. Well, this copy is not usable for a recover. 
+The Inode is deleted, but perhaps there is also still an even older copy of 
+the same data block.
+If you find such an older block in the Journal, then you can find there the old
+intact Inode copy of the deleted file. And with such an old Inode, you can now
+undelete the file. You find in the Inode the properties and all refer to the 
+data blocks. In the directory you find the old file name.  With a little luck,
+the data blocks are not reused.
+
+This is the principle of ext4magic to recover from inode copies. 
+
+In the Journal there are not only inode copies.  You will also find tables with
+the block and inode allocation. This data are used in the magic functions
+for controlling the file carving. The functions of the file carving matched
+exactly to the respective properties of the file system types and these functions
+included into a multi-stage recover process. This feature is new in this version
+and currently only usable for ext3.
+
+
+
+----------------------------------------------------------------------------------
+
+
+How you can use ext4magic ?
+===========================
+
+You need, of course, the file system from which you want try to recover deleted 
+files. The safest way is to create an image of the partition. 
+Important, for this, the filesystem must umounted or readony mounted.
+
+For example: the filesystem is on /dev/sda1
+
+ # dd  if=/dev/sda1  of=/path/to/image_name  bs=4096
+
+
+With the shell, you change to a directory, where enough free space to write 
+the data recovers.  You need also some options, but that later. 
+
+You can use ext4magic:
+
+ # ext4magic  /path/to/image_name  options
+
+
+
+
+Not enough free space for a imagefile of the entire filesytem ?
+-------------------------------------------------------------
+
+If you can use ext4magic from a rescue system or from a LINUX Live-system,
+or on a other LINUX system, do not mount the partition with the deleted files and use it directly 
+
+  # ext4magic /dev/sda1 options
+
+
+
+
+You can not restart the computer or umount the partition ?
+---------------------------------------------------------
+
+Attempts to mount the partition readonly. The best way try to "umount" and then 
+"mount -o ro /dev/sda1" . If this ist noch posible?  try the following:
+
+ # mount -o remount,ro,noload  /dev/sda1
+
+if the partition is now mounted readonly, use also
+ # ext4magic /dev/sda1 options
+
+
+
+
+It is impossible to mount readonly ? 
+------------------------------------
+
+ext4magic still has a solution, but highly experimental. Please use only in 
+exceptional cases. Never use ext4magic for a not readonly mounted partition. 
+ext4magic read over the filesystem buffer from journal but the kernel write 
+unbuffered to journal.
+
+The first read of the journal is often ok, but all subsequent reads can read
+wrong data blocks from journal. So long the journal file is buffered, you read 
+wrong data blocks at the moment of the first read. The file system is operating
+normally without errors, but ext4magic reads wrong blocks from the Journal.   
+
+
+Workaround : ext4magic supports external journal. 
+You can create a copy of the filesystem journal with the "debug2fs" command. 
+Use this copy as external Journal for the mounted file system.
+But, if mounted readwrite, here also only the first backup will work good,
+after read the journal by debug2fs, the journal is also buffered and the next
+read by debug2fs results also a bad journal copy.
+
+
+  # debug2fs  -R  "dump <8> /path/to/journalbackup" /dev/sda1
+
+you can use this copy of Journal  
+
+  # ext4magic /dev/sda1 -j  /path/to/journalbackup options 
+
+ext4magic then only read journal data from this journal backup. 
+  
+Warning: This procedure is tested, it works, but please be very careful 
+with this feature. Remember, for ext4magic the file system is frozen at the time at
+which the journal copy created. Any subsequent changes will not recognize by ext4magic.
+This works only for a limited time if you continue to write into the file system.
+
+
+
+
+--------------------------------------------------------------------------------------------------
+
+
+A few words about the new magic functions (current version 0.2.0)
+ ============================================================
+ These functions are designed to make undo of recursive deletes. It is a  multi-level recover
+ and also restore files if no old journal copies can be found for this file.
+
+
+   1. recover files of the file system tree with the help of old inode copies.
+   2. recover all other inode copies which were not found in first stage. 
+   3. (currently only ext3) recover the remaining data blocks, using a file carving function (we say magic function)  
+ 
+
+
+ After an accidental deletion:  prevent all writing into this file system and if possible also
+ prevent reading of this file system. Also reading overwrites old journal data
+ which are needed for the restore. 
+
+ Umount the file system, and use ext4magic before you mount the file system again,
+ or create a copy of the file system and use this for the recover.
+ Perform no file system check on this file system before.
+
+
+ The magic functions are very user friendly because very few command options are required.
+
+ Extensive testing has confirmed that magic-scan-functions are now stable with libmagic of file-5.04.
+ Good support exists for: all text file types, a lot of image formats, 
+ often-used video and audio file types, Open Office documents,
+ PDF, RAR, TAR, CPIO, BZ2, ZIP, GZIP, 7Z ...
+ 
+ Many other file types are also found and restored with default function, but without examining
+ the contents of the files. This works more or less. 
+
+ Problems still exist with some multimedia formats and some documents. Not every file type
+ can be restored only based on head and foot patterns. Some types of multimedia streams, splited or
+ truncated files are hard to recover.
+ The recovery of CD/DVD images and other file system containers is also problematic. This can only work 
+ in file systems with 4KB block size.
+ Sparse files, and very large files if not deleted in one step, can not be restored with this 
+ function. (Bug:#017607)
+ Of course, you can only find files when the "file" command recognize this file type. It is theoretically
+ possible to enable the restore of unknown file types by an entry in the configuration file to "magic".
+ 
+
+ Some files are one (or few) byte too short. These are final zero byte.
+ Most of these files can be repaired by appending zeros.
+ The following command illustrates how attach two zero byte to a file.
+
+ #echo -en "\0\0" >> file
+
+
+ Some files are one or more bytes to long. These are often zero byte at the end of the restored file.
+ You can see this at the end of a file. "hexdump-C file | tail "
+ These files can be opened usually normal, possibly with a warning. Only a few programs block the
+ processing of such files.  Here is an example,  how this can be fixed (xz compressed file)
+
+# ls -l test.xz
+-rw-r--r-- 1 rob users 1005  4. Dez 12:54 test.xz
+# xz -t test.xz
+xz: test.xz: Compressed data is corrupt
+# xz -d test.xz
+xz: test.xz: Compressed data is corrupt
+# dd if=test.xz of=test_.xz bs=1 count=1004
+1004+0 Datensätze ein
+1004+0 Datensätze aus
+1004 Bytes (1,0 kB) kopiert, 0,0164605 s, 61,0 kB/s
+# xz -t test_.xz
+# xz -d test_.xz
+
+ 
+ The magic functions do not work particularly fast, but very efficient and can find some files
+ that other tools can not recover. It also find very long files when the data are fragmented in the
+ file system. Others file carving tools find here often no complete files, or recover data trash.
+ Because of the previously running recover stages, the hit rate of this function is often very good.
+ But, at very high fragmentation the chances are low for a successful recovery for many files.
+ 
+ In real file systems the magic function find also unfortunately some very old files.
+ The idea, to prevent this by using the metadata from the journal, is definitely good, but, 
+ in a real file system it works only limited. In test file systems it works very well, but in a real
+ file system journal you find not always enough of these metadata to prevent the recover of very old files.
+
+ 
+
+--------------------------------------------------------------------
+
+
+
+Instructions to experimenting with new features, the magic functions.
+---------------------------------------------------------------------
+
+Use no file system specially created for this purpose.
+Why?
+ If you create a test file system, it is likely that all inode copies are included
+ in the Journal. The first stage can restore all files, and you'll never see the
+ magic functions in the third stage.
+
+
+Better is the following:
+ Use an existing ext3 filesystem. The last hours should no run a global "find" or a backup tool
+ in this file system. That too would write to many inode copies and to be easy to recover.
+ umount this file system, and create a 1-to-1 copy of the file system.
+ Now mount the file system copy and delete all or many files. Then umount the file system copy.
+
+
+
+Now you can test ext4magic with the deleted copy.
+ You need free space for writing the recovered files.
+ Assuming, the copy is "/dev/sdb1" and you have enough free
+ space at "/home/test/"
+
+ # ext4magic /dev/sdb1 -d /home/test/RECOVER -M
+ if you have deleted all files.
+
+ or
+ # ext4magic /dev/sdb1 -d /home/test/RECOVER -m
+ if not all files were deleted.
+
+
+ It will automatically search for the time of the last deletion.
+ And with a little delay should start the recover. You can now only wait. Depending on the
+ number of deleted files can take a long time. Then you can compare the files with 
+ the original file system.
+
+
+--------------------------------------------------------------------------------------------
+
+The Options of ext4magic
+=========================
+
+ext4magic has a lot of options, here are just a small overview.
+Detailed information take from the manpage.
+
+One option must always be specified, the file system.
+
+
+Information Options  -S -J -H -T 
+---------------------------------
+This can display information from the file system, the journal ore the transactions from journal.
+
+
+
+Selections  -I -B  -f
+-----------------------
+select the specific inode, blocks or file names for the information- and action options.
+
+
+
+Time Options -a -b -t
+------------------------
+These are important control options. This indicates the time window for searching for files.
+
+
+
+File input and output options  -d -i -j
+---------------------------------------
+This can be specified, the output directory, a input file list and an external journal file
+
+
+
+Action Options  -l -L -r -R -m -M
+----------------------------------
+For select of the various listing- and recover actions.
+
+
+
+
+---------------------------------------------------------------------------------------------
+
+
+Some common problems
+
+Command not found 
+------------------
+ext4magic is installed to /usr/local/sbin/ 
+This directory is only included in the PATH if you use root as a login shell.
+For a full root environment  use "su -l" for the user change.
+
+
+
+Manpage not found
+-----------------
+The manpage is installed under /usr/local/*/man/man8/
+Check if the MANPATH variable include the following directories.
+
+ /usr/local/man /usr/local/share/man
+
+
+
+ext4magic nothing works
+-----------------------
+two possible causes: 
+- either you are not root
+- or the time options are not set correctly. Only the magical functions automatically search 
+  for the best time window, all other options use default time values. (See manpage) 
+
+
+
+----------------------------------------------------------------------------------------------
+
+Known Bugs
+
+Only on big endian environments, there are some incorrect outputs of time stamps, and missing of
+deleted directory entries. (BUG #017304 #017305) 
+These errors occur only if the journal is not read and so only called functions of libext2fs for
+printout of inode and directorys. All journal options and the file restoring are not affected. 
+The error is not within ext4magic and can not be compensated in ext4magic. This would be patched
+in libext2fs. The error is very rare and not significant. If anyone needs a patch for this,
+no problem, within ext4magic the problem is solved. It is also possible to write an unofficial patch
+for libext2fs. I just think that nobody will really need it. Otherwise, send a request to the ext4magic 
+mailing list.
+