help with DAT database - urgently

[msfeline]

Hi

Recently, I inherited a project to revamp one of my company's old system.
Currently, we are using this software called The Computer Aided Records Management System (CARMS, company bought in 1990s) and according to the manual, CARMS was developed in "C" language. The database is in a DAT format but the data is not in ascii form and rather non-decipherable. I need to find a way to extract the contents out for migration purposes and I simply have no clue how. No one else that I approached is able to do it either.

Anyone got any clue or any suggestions so I can get started?
Any one has any extraction program?

[Paft]

If it's not in plain ASCII, it's physically impossible to just read back the data. Sure, the people could have stored it in plain binary format, which would be readable by just writing a binary->ASCII converter.. but if they even put the most simple encryption or compression on the .dat file, you're out of luck exporting the data unless they explicitly wrote a way to do so OR you can reverse-engineer how the data is stored.

[msfeline]

that sound so hopeless..

but the thing is after i analyse it more thoroughly, its the dates field that are garabage,, the text field seems ok... any idea?

[in2deep]

Dates are often held as the number of second or milliseconds since a reference date, and stored as a binary number. This is usually a 32 bit value. Thus, if you are seeing dates changing to 4 bytes of "garbage" this is probably what you are seeing. Create 2 records a known interval apart and subtract the lower number from the higher, this should give you an idea of what the resolution of the timer is.

As to what the base date is, more difficult because there are several common standards, but armed with the counters significance, it should be possible to calculate the reference date.

The number of Milliseconds since 24:00 GMT, Jan. 1, 1970 is a common standard.

Do you have the source code for the CARMS system? If so, reverse engineering it's storage mechanism should be easy. If the data files are simple, even without the source, it should be a straight forward task with a decent hex editor and some C expertise. Definitely not hopeless.

[Paft]

that sound so hopeless..

but the thing is after i analyse it more thoroughly, its the dates field that are garabage,, the text field seems ok... any idea?

Ohhh, so this is datetime. A'right.. as was said above, if this is 4 bytes wide binary, then you're probally looking at UNIX time, which is based off Jan 1, 1970. Your best bet for finding out what sort of intervals you're looking at would be to subtract one date from another and figure out the 4-bytes' difference. That will give you an interval starting point. If you have a way to retrieve the dates, even better. Convert the date in words to an interval, a common one is seconds, and then convert that to hexidecimal. Compare that hex value against the hex value you get with a hex editor looking at the .dat file, and if they're the same, it's a straight conversion. If they're different, it'll take a little work figuring out how the dates were hashed.

[msfeline]

ok, other questions aside.
Can i check what are the common DAT databases used in DOS programming? i now understand that the one i am dealing may not be propietary. thanks guys

[in2deep]

Wotsit shows only one proprietary database with a .dat type file, DataFlex, one that I am unfamiliar with.

The thing to be aware of is "database" is a highly abused term. Many authors who store a few records in a file will refer to the file as a database, when it is simply a file. Most true database systems use either multiple files, or a partitioned single file.

Probably the most common file extension given to these "files" is "something.dat" simply referring to the "data" of "something".

Do you have the original source of CARMS, or have you simply executables and data files? If you have the source, working out the file format will be easy of course.

[msfeline]

Wotsit shows only one proprietary database with a .dat type file, DataFlex, one that I am unfamiliar with.

The thing to be aware of is "database" is a highly abused term. Many authors who store a few records in a file will refer to the file as a database, when it is simply a file. Most true database systems use either multiple files, or a partitioned single file.

Probably the most common file extension given to these "files" is "something.dat" simply referring to the "data" of "something".

Do you have the original source of CARMS, or have you simply executables and data files? If you have the source, working out the file format will be easy of course.

yes, i think u are right. its simply files with DAt extension. No, i do not have the source code for CARMS cos its a off the shelf software

[in2deep]

Going back to your original post, you say you need to extract the data from the file. Can you not use your existing CARMS software to do that? For example, can you "print" the data to a file for example? How big is this file?

[A_old]

Going back to your original post, you say you need to extract the data from the file. Can you not use your existing CARMS software to do that? For example, can you "print" the data to a file for example? How big is this file?

exactly my suggestion..just use the carms software to get the data manually and enter it into the new system...might be a pain..but u get the information. gl w/ it

[in2deep]

I'm guessing he wants to automatically transfer this data from where it is now to some other system. If the amount of data is small, then a manual transcription will be boring, but may use less resources then writing something.

If the volume of data is large, then my suggestion of printing to a file will resolve his data translation issues as the CARMS software will do this for him. Without more details, it is difficult to offer concrete help.