DefleMask Forum

DefleMask => General => Topic started by: Delek on February 04, 2013, 03:23:52 am

Title: .DMF Specs
Post by: Delek on February 04, 2013, 03:23:52 am
If you are writing a music app and you want to include support for DMF format, you want to convert any format to the DefleMask format or transform a DMF to a different filetype, this will be helpful:

.dmf
LATEST DMF FORMAT SPECS (http://www.deflemask.com/DMF_SPECS.txt)

OLD ONES:
http://www.deflemask.com/DMF_SPECS_0x15.txt
http://www.deflemask.com/DMF_SPECS_0x13.txt
http://www.deflemask.com/DMF_SPECS_0x12.txt
http://www.deflemask.com/DMF_SPECS_0x11.txt

.dmf
LATEST DMP FORMAT SPECS (http://www.deflemask.com/DMP_SPECS.txt)

OLD ONES:
http://www.deflemask.com/DMP_SPECS_0x0A.txt
http://www.deflemask.com/DMP_SPECS_0x09.txt
Title: Re: .DMF Specs
Post by: UltrasonicMadness on February 11, 2013, 12:58:37 pm
I don't claim to be an expert in file formats but I think you may have made a typo here:
Quote
SYSTEM_SMS 3         (SYSTEM_TOTAL_CHANNELS 3)
Last I checked, the SMS had 4 channels.

You even have the Mega Drive/Genesis listed with 10 channels, which I would assume is 6 for the YM2612 and 4 for the TI SN76489.

I apologize if you have used some compression sorcery resulting in this point being null and void and me appearing ignorant.

As an aside, do you know how I can decompress DMF data so I can look at what, for instance, Monday looks like in a hex editor?
Title: Re: .DMF Specs
Post by: Delek on February 11, 2013, 01:33:00 pm
You are right, I will fix that.

You should use Zlib library to decompress DMFs.
Title: Re: .DMF Specs
Post by: UltrasonicMadness on February 11, 2013, 03:30:02 pm
I checked www.zlib.net (http://www.zlib.net) and all I could find was source code.
Title: Re: .DMF Specs
Post by: r57shell on February 15, 2013, 10:23:53 am
If it will help, I use this programs on python.
Compress:
Code: [Select]
import zlib, sys
str_object1 = open(sys.argv[1], 'rb').read()
str_object2 = zlib.compress(str_object1, 9)
f = open(sys.argv[2], 'wb')
f.write(str_object2)
f.close()
Decompres:
Code: [Select]
import zlib, sys
str_object1 = open(sys.argv[1], 'rb').read()
str_object2 = zlib.decompress(str_object1)
f = open(sys.argv[2], 'wb')
f.write(str_object2)
f.close()
Usage (from command line): file.py input output

If you have some simple way to support zlib compression in C++ source, please post here :).
Title: Re: .DMF Specs
Post by: UltrasonicMadness on February 18, 2013, 08:04:38 pm
Sorry if I sound like a complete n00b but how would I 'chdir' or 'dir' in Python?
Title: Re: .DMF Specs
Post by: r57shell on February 19, 2013, 10:22:52 am
Code: [Select]
os.chdir("/path/to/change/to")compress.py supports full paths.
from C++ you can start process on python for example
Code: [Select]
#include <cstdio>
#include <cstdlib>

void Compress(char* input_path, char* output_path)
{
    char cmd[255];
    sprintf(cmd, "compress.py \"%s\" \"%s\"", input_path, output_path); // make cmd line
    system(cmd); // execute cmd line (start process)
}
Title: Re: .DMF Specs
Post by: r57shell on February 19, 2013, 11:14:51 am
For MIDI to DMF tool, I want to requset speed configuration into pulses per beat, and BPM calculation algorithm.
More definitely:
BPM = how many Beats in minute.
Pulses per beat = how many Pulses in beat.
Pulse = row in my opinion.
So BPM*(Pulses per beat) = how many rows in minute. It is more simple(clear) thing than speed.
Title: Re: .DMF Specs
Post by: UltrasonicMadness on February 19, 2013, 04:59:59 pm
I tried it and this is what I got

Code: [Select]
>>> os.chdir(C:)
SyntaxError: invalid syntax
>>>

I'm running Python 2.7.3 with the IDLE GUI on Windows XP Service Pack 3.

Regarding your more recent post, I'm probably not going to write anything based on this. I just thought it would be interesting to see an uncompressed DMF file in a hex editor side-by-side with Delek's specifications.
Title: Re: .DMF Specs
Post by: Delek on February 19, 2013, 06:39:52 pm
Code: [Select]
char *dst=NULL, a=0, *src=NULL;
unsigned long dstSize=0, srcSize=0, i=0;
FILE *DMFFILE=fopen("dmf.dmf", "rb");

fseek(DMFFILE, 0, SEEK_SET);
while(!feof(DMFFILE)){
    fread(&a,1,1,DMFFILE);
    srcSize++;     //GET FILE SIZE
 }
 dstSize=srcSize*256;       //MAX DECOMPRESS RATIO
 src=malloc(srcSize);
 dst=malloc(dstSize);
 if(src!=NULL && dst!=NULL){
     fseek(DMFFILE, 0, SEEK_SET);
     while(!feof(DMFFILE)){
         fread(&src[i],1,1,DMFFILE);
         i++;
     }
     uncompress(dst, &dstSize, src, srcSize);
     fclose(DMFFILE);
     DMFFILE=fopen("tmp", "wb+");
     fwrite(dst, 1, dstSize, DMFFILE);
 }
 free(src);
 free(dst);

With this code you will have an uncompressed DMFfile called "tmp".
Title: Re: .DMF Specs
Post by: r57shell on February 20, 2013, 12:15:58 pm
I tried it and this is what I got
:-\

WinXP/Win7:
1) Hold WIN+R, window opened, Release WIN+R  :D
2) Type cmd in opened window, press ENTER, new window appears. This - named "command line".
3) Now, drag & drop decompress.py saved somewhere on your system into this black (by default) window - decompres.py full path inserted,
4) Type space (one character), and make sure that it is appeared after file path.
5) Now, drag & drop file which you want to decompress - full path appeared
6) Type space (one character), AGAIN
7) Now, drag & drop file which you will replace (for example just clean bin file with size = 0) - it will be resulting uncompressed file.
8) press enter, in command line, to execute comand.
All you need is: Python (for example 2.7.3), and saved compress.py on hard drive.
If it won't work, try to append in begining of command line "python " (without quotes).

It is just 3 paths, delimeted with space, as I told
Usage: decompress.py input output

Delek, main question - where to get "uncompress" function?)
Title: Re: .DMF Specs
Post by: Delek on February 20, 2013, 05:34:17 pm
Uncompress function is, obviously, inside ZLIB library!
Title: Re: .DMF Specs
Post by: r57shell on February 20, 2013, 09:18:28 pm
Delek, Thanks. It's works. Now my code using zlib compression internaly.

If it will help, there is source of my MemFile to simplify operations with zlib. Just replace FILE *f = fopen(...), to MemFile*f = new MemFile(); And use f->pack(), for packing after all data writed.
http://pastebin.com/Q8kmPj0D (http://pastebin.com/Q8kmPj0D)
Title: Re: .DMF Specs
Post by: Delek on February 20, 2013, 10:40:59 pm
Great, thanks!
Title: Re: .DMF Specs
Post by: UltrasonicMadness on February 21, 2013, 03:00:57 pm
It worked! I apologize for being such a n00b. I thought I had to do the whole thing in IDLE.

I gotta say, the DMF format looks interesting...
Title: Re: .DMF Specs
Post by: Alriightyman on April 06, 2013, 12:07:12 am
I have a few Questions.  (These are specifically for the Sega Genesis)
1. In the WAVETABLES data, if the TOTAL_WAVETABLES is 0, are the next 4 bytes still considered and in the file?  Or does it immediately go into patten data?
2. What are the NOTE values?  Would C#-0  be 0?
EDIT:   I think I figured out the notes.
C  - 1, C# - 2, D  - 3, D# - 4, E  - 5, F  - 6, F# - 7, G  - 8, G# - 9, A  - A, A# - B, B  - C
So, my new question is what is 0?  Is is the same as the last note played?
3. Is Octave value 0 - 8?  Also, what are Effect code values( Not Effect Value )?
4. Are LFO and LFO2 the same as FMS and AMS?
5. RS is listed in the program but not in the file format for FM instruments. I am assuming KSR is the equivalent.  Am I right?  Also, is KSR supposed to be listed twice in the FM istrument format?

I am thinking about writing a program to convert from dmf to the genesis SMPS format.  But these few things are stopping me.
I appreciate any help thanks!
Title: Re: .DMF Specs
Post by: Delek on April 06, 2013, 01:51:18 am
1.
As the doc says:
* 1 Byte: TOTAL_WAVETABLES
* Repeat this TOTAL_WAVETABLES times
If there is 0 TOTAL_WAVES, all inside the block "REPEAT THIS" disappear. It immediately go into pattern data.

2. Thanks for reporting that, I forget to add the note values list. The list will be added on the file format now:
NOTES:
01 C#
02 D-
03 D#
04 E-
05 F-
06 F#
07 G-
08 G#
09 A-
10 A#
11 B-
12 C-

Special cases:
Note = 0 and octave = 0 means NO NOTE.
Note = 100 means NOTE OFF, no matter what is inside the octave value.

3. Octave value can go from 0 to the system max_octave value. Some systems has more octave range than others.
Effect code is the selected effect, for example:
0B = POSITION JUMP, then 0B 02 means POSITION JUMP to PATTERN 02.
EFFECT CODE, EFFECT VALUE
0B 02
(http://images.wikia.com/deflemask/images/b/b7/0Bxx1.png)

4. Yes, FMS = LFO, AMS = LFO2. I will add this on the specs.

5. Yes, it is KSR. The first KSR is a reserved value and must be 0, it was a type error. Fixed.


All issues are corrected, please reload the specs file. Thanks.
Title: Re: .DMF Specs
Post by: FlashingYoshi on April 19, 2013, 11:03:12 pm
I am thinking about writing a program to convert from dmf to the genesis SMPS format.  But these few things are stopping me.
I appreciate any help thanks!
I honestly would appreciate this very much!
Title: Re: .DMF Specs
Post by: CountSymphoniC on October 10, 2014, 03:34:59 pm
Hey Delek, I sent you a PM over at Chipmusic in regards to adding support for Deflemask's standard for Tracker commands/effects, and another tidbit of a request. Good day.
Title: Re: .DMF Specs
Post by: Delek on October 12, 2014, 01:02:38 pm
Replied, thanks.
Title: Re: .DMF Specs
Post by: Kabuto on March 08, 2015, 10:11:32 pm
Some issues I found when parsing a Genesis file after reading the specs (version 10c for both specs and tracker):

* FM instruments all have TOTAL_OPS == 0, and there are 4 operators present, and not 2 like the docs claim
* The loop position is omitted in empty envelopes
* There's an extra zero byte at the song end (in the decompressed stream) that's not documented
* Are character sets just plain ASCII and no specific character set? I guess yes since the GUI won't accept non-ASCII characters
Title: Re: .DMF Specs
Post by: Delek on March 08, 2015, 11:23:29 pm
Thanks Kabuto.
* FM instruments all have TOTAL_OPS == 0, and there are 4 operators present, and not 2 like the docs claim
Yes, actually if a FM soundchip has only ONE possible setup of total operators, that byte is useless. It is there just for systems like the YMU759 and other OPL soundchips capables of set the operator numbers. DefleMask only uses that byte on YMU759 at the moment.
I will add this information in the specs.
* There's an extra zero byte at the song end (in the decompressed stream) that's not documented
Strange. The samples count maybe? Try to load a sample, that number should be 1/2/3 and after that the sample stream data.
* Are character sets just plain ASCII and no specific character set? I guess yes since the GUI won't accept non-ASCII characters
Yes, this can be found in the very first page of the DefleMask's manual.
"* Current DefleMask character support: ASCII"
Title: Re: .DMF Specs
Post by: Kabuto on March 09, 2015, 12:37:35 am
* There's an extra zero byte at the song end (in the decompressed stream) that's not documented
Strange. The samples count maybe? Try to load a sample, that number should be 1/2/3 and after that the sample stream data.
Just re-checked, this was with samples loaded, it's clearly a byte that comes after sample data. Though I have used Java's built-in InflaterInputStream for decompression, maybe that plays a role.
Title: Re: .DMF Specs
Post by: cigueso on January 05, 2016, 12:24:32 pm
Hi,

I am currently writing a DMF file parser/reader, and unless I did something wrong,
it seems like there is one undocumented byte between TOTAL_ROWS_IN_PATTERN_MATRIX and PATTERN MATRIX VALUES.

I am using DefleMask v11 on Mac. On both the demo songs Last Battle and Magician Domain (PC Engine) this byte is equal to 1.
Thank you for any information about this.
Title: Re: .DMF Specs
Post by: Delek on January 05, 2016, 12:46:24 pm
Sorry, that value was removed, it was the Arpeggio Tick speed and now it is an effect-controlled-value rather than a global one. Specs are for version 11.1 and the module you are trying to open is for version 11.

I added a note in the specs to clarify this, thanks for the report.
Title: Re: .DMF Specs
Post by: cigueso on January 05, 2016, 01:16:56 pm
Please dont be sorry, thank YOU for the quick answer and this great tracker !
( if only for the SMS support i was always hoping for :) )
Title: Re: .DMF Specs
Post by: tildearrow on January 06, 2016, 03:42:54 pm
May I ask where can I get the old specs with YMU759 information?
Title: Re: .DMF Specs
Post by: Delek on January 06, 2016, 04:28:29 pm
Why do you need them? It should be on my old backups, but you can open a YMU759 .dmf with an old DefleMask version, edit the patterns, change the system and save it to open it in the new Defle.
Title: Re: .DMF Specs
Post by: tildearrow on January 06, 2016, 06:21:47 pm
Because I'm writing a program which exports to another format and I want it to support YMU759.
And although I could, I don't think I'll be able to fit 17 channels in 10...
Title: Re: .DMF Specs
Post by: LeCheeseMaster16 on January 06, 2016, 06:36:59 pm
Someone should really create a program that can rip game music to .dmf files. For example, Sega genesis game? it will be placed into a file that has all the ripped sega genesis music on that game. Same as gameboy/SMS/NES/Gameboy/Commondor 64. idk if it might be a good idea, but if someone makes that type of program, I would love it instantly. If its out on beta, can you PM me the link for that program?
Title: Re: .DMF Specs
Post by: tildearrow on January 06, 2016, 08:16:18 pm
Something isn't right here...
http://i.imgur.com/AYaYUWs.png (http://i.imgur.com/AYaYUWs.png)

In the Instruments part of a Genesis DefleMask file, in the FM part of any instrument there should be 52 bytes (4 for the header and 48 for the oscillators). Why do we have some extra bytes? Are they undocumented or remainders from older versions?
Title: Re: .DMF Specs
Post by: Delek on January 06, 2016, 08:32:02 pm
That file has VERSION_NUMBER 0x12 (really old), so you will find, for sure, surprises if you use a specs from version 0x14 (v11.1)
Your parser needs to check this and shouldn't accept that file.

EDIT:
Here you have some old specs if you want, I recovered them from backups:
http://www.delek.com.ar/soft/deflemask/DMF_SPECS_0x11.txt
http://www.delek.com.ar/soft/deflemask/DMF_SPECS_0x12.txt
http://www.delek.com.ar/soft/deflemask/DMF_SPECS_0x13.txt
Title: Re: .DMF Specs
Post by: tildearrow on January 06, 2016, 08:36:13 pm
Now it was all about the reserved bytes. Thank you SO much!!  :D
Title: Re: .DMF Specs
Post by: Delek on January 06, 2016, 08:42:12 pm
You're welcome, check out the EDIT of my previous post (I added 0x12 version specs)
Title: Re: .DMF Specs
Post by: tildearrow on January 17, 2016, 04:27:13 am
Oh and umm... System number 71 (which is Commodore 64 with the 6581 chip) is not documented...
Title: Re: .DMF Specs
Post by: Delek on January 17, 2016, 11:33:26 am
Yeah, I have to update it with "modes" in the system. That is the most significant nibble in the SYSTEM byte, used in C64 and Genesis with Ext.CH3 mode.
Title: Re: .DMF Specs
Post by: Pegmode on March 19, 2017, 12:22:23 am
Hey could we by chance get the .dmw specs? I've pulled them apart and understand the relatively well but I've seen some big inconsistencies in between the demo .dmw files and custom ones generated by Deflemask. I'm currently working on a project that requires this knowledge.

Demo .dmw vs Deflemask generated .dmw:
(http://i.imgur.com/ySCr5pv.jpg)
Title: Re: .DMF Specs
Post by: SeregaZ on January 17, 2018, 03:38:59 pm
i am not understand how unpack it too :) my old code, that unpack VGZ to VGM work fine. i try to use same for DMF... and nothing. you sure it was zlib? i thought i am do something wrong and download zlib unpacker program - it cant to unpack too. any 7z can unpack VGZ, but they cant unpack DMF. it some conspiracy :) for VGZ - i am read last 4 bytes - it have unpacked size of file. it need for allocate memory for unpacked data. when i try to read same last 4 bytes - DMF have negative value. it is a little strange.

maybe you can make some options for save? i mean save as normal - packed, and save as unpacked for user choice for save.
Title: Re: .DMF Specs
Post by: cyberic on January 17, 2018, 08:57:49 pm
Oh and umm... System number 71 (which is Commodore 64 with the 6581 chip) is not documented...
Is there an existing vgm player for c64 .vgm files?
Title: Re: .DMF Specs
Post by: SeregaZ on January 18, 2018, 12:11:31 pm
cyberic, probably this chip is not support by vgm. i didnt see it in specification: http://vgmrips.net/wiki/VGM_Specification

****************

how to understand this part?
Code: [Select]
//SYSTEM SET
1  Byte:  System:
  SYSTEM_GENESIS 0x02 (SYSTEM_TOTAL_CHANNELS 10)
  SYSTEM_GENESIS (EXT. CH3) 0x12(SYSTEM_TOTAL_CHANNELS 13)
  SYSTEM_SMS 0x03 (SYSTEM_TOTAL_CHANNELS 4)
  SYSTEM_GAMEBOY 0x04 (SYSTEM_TOTAL_CHANNELS 4)
  SYSTEM_PCENGINE 0x05 (SYSTEM_TOTAL_CHANNELS 6)
  SYSTEM_NES 0x06 (SYSTEM_TOTAL_CHANNELS 5)
  SYSTEM_C64 (SID 8580) 0x07 (SYSTEM_TOTAL_CHANNELS 3)
  SYSTEM_C64 (SID 6581) 0x17 (SYSTEM_TOTAL_CHANNELS 3)
  SYSTEM_YM2151 0x08      (SYSTEM_TOTAL_CHANNELS 13)

in my case this value is 0x42 i thought to see 0x02 - sega mega drive. what the 0x42???
Title: Re: .DMF Specs
Post by: cyberic on January 18, 2018, 12:44:15 pm
cyberic, probably this chip is not support by vgm. i didnt see it in specification: http://vgmrips.net/wiki/VGM_Specification
Yes because ValleyBell and Delek disagreed...
See http://vgmrips.net/forum/viewtopic.php?f=3&t=342&start=0 (http://vgmrips.net/forum/viewtopic.php?f=3&t=342&start=0)
I was asking because I wondered if the developers had used a specific tool to test the .vgm output of a c64 system
Title: Re: .DMF Specs
Post by: SeregaZ on January 19, 2018, 07:14:31 am
still didnt understand what the //SYSTEM SET 0x42 value? it says 0x02 or 0x12 must be for sega genesis.

and what a ext 3??? what a genesis support that? where do you get 13 channels? if always was 10. FM 6 and PSG 4. i know special mode for third channel Ch3mode, but it is not 4 additional channels. it is one channel.
Title: Re: .DMF Specs
Post by: Hpman on July 10, 2018, 12:22:48 pm
Seconding previous post, Genesis ext.3 mode is documented as 0x12 but is indeed 0x42 in the file (DM 0.12.0)
Title: Re: .DMF Specs
Post by: nitrofurano on December 14, 2018, 11:52:27 pm
how do you expand these gzip-compressed files? i?m trying with
 cat tune.dmf | gzip -dc > tune_expanded.dmf
and having no success at all
Title: Re: .DMF Specs
Post by: Cicero on January 04, 2019, 10:53:21 pm
i am not understand how unpack it too :) my old code, that unpack VGZ to VGM work fine. i try to use same for DMF... and nothing. you sure it was zlib? i thought i am do something wrong and download zlib unpacker program - it cant to unpack too. any 7z can unpack VGZ, but they cant unpack DMF. it some conspiracy :) for VGZ - i am read last 4 bytes - it have unpacked size of file. it need for allocate memory for unpacked data. when i try to read same last 4 bytes - DMF have negative value. it is a little strange.

maybe you can make some options for save? i mean save as normal - packed, and save as unpacked for user choice for save.
how do you expand these gzip-compressed files? i?m trying with
 cat tune.dmf | gzip -dc > tune_expanded.dmf
and having no success at all
I'm also having trouble here. I was trying to write a script in Kotlin to parse DMF files (and then maybe do something more useful with them), but whenever I tried using the Java standard library to decompress, I'd hit exceptions talking about incorrect headers or invalid stored block lengths.

If there was an option to save uncompressed, that would also work, although ideally maybe we'd figure out what's causing us to have trouble decompressing.
Title: Re: .DMF Specs
Post by: SeregaZ on January 07, 2019, 05:50:12 pm
o! someone is a live here? :) it cant be! if you still interest i can make some "unpacker"... anyway i have some problem with someone's Last Christmass song :) by some reson it read as 0 instruments and my converter dmf2gems is stuck. probably i need to dig deeper. heh... i thought i will make for a few minuts that convert and start to record video - but now i am see it will not that few minuts :) converting from midi is crap - i have not synchronous and my instruments is bad. so i thought make convert it from deflemask... and it is terrible :) need to update whole code :)
Title: Re: .DMF Specs
Post by: SeregaZ on January 07, 2019, 06:25:01 pm
a few moment later it was done. not tested with windows 7 and higher - XP my everything :)