Skip to main content
  • KSP Reference Manual
  • Music Information Retrieval

Music Information Retrieval

General Information

Music Information Retrieval (MIR) allows the extraction of meaningful features from audio files, such as pitch or the volume level of a sample. New KSP commands allow extraction of such parameters from samples via script. MIR functions are not asyncronous in the on init callback, but in all other callbacks they will run asynchronous.

Note: the type detection functions listed below (Sample Type, Drum Type, and Instrument Type) are designed to process one-shot audio samples.

detect_pitch()

detect_pitch(<zone-id>, <pitch-result>)

Returns a real value representing the fundamental frequency of an audio sample, in semitones and cents. If detection fails, the function will return ~NI_DETECT_PITCH_INVALID.

<zone-id>

The ID of the zone.

<pitch-result>

The MIDI note value of the detected pitch.

detect_loudness()

detect_loudness(<zone-id>, <loudness-result>)

Returns a real value representing the loudness of an audio sample in decibels. Loudness is measured according to the standard established by the International Telecommunication Union: Algorithms to measure audio program loudness and true-peak audio level - ITU-R BS.1770-4 (2015). If detection fails, the function will return ~NI_DETECT_LOUDNESS_INVALID.

<zone-id>

The ID of the zone.

<loudness-result>

The real value of the detected loudness in decibels.

detect_peak()

detect_peak(<zone-id>, <peak-result>)

Returns a real value representing peak level of an audio sample in decibels. Peak is measured according to the standard established by the International Telecommunication Union: Algorithms to measure audio program loudness and true-peak audio level - ITU-R BS.1770-4 (2015). If detection fails, the function will set <peak-result> to ~NI_DETECT_PEAK_INVALID.

<zone-id>

The ID of the zone.

<peak-result>

The real value of the detected peak level in decibels.

detect_rms()

detect_rms(<zone-id>, <rms-result>)

Returns a real value representing the RMS level of an audio sample in decibels. If detection fails, the function will return ~NI_DETECT_RMS_INVALID.

<zone-id>

The ID of the zone.

<rms-result>

The real value of the RMS level of the audio sample in decibels.

detect_sample_type()

detect_sample_type(<zone-id>, <sample-type-result>)

Assigns <sample-type-result> a $NI_DETECT_SAMPLE_TYPE tag describing the whether an audio sample is a drum or an instrument. If detection fails, the function will return $NI_DETECT_SAMPLE_TYPE_INVALID.

<zone-id>

The ID of the zone.

<sample-type-result>

The detected sample type, can be one of the following:

$NI_DETECT_SAMPLE_TYPE_INVALID

$NI_DETECT_SAMPLE_TYPE_INSTRUMENT

$NI_DETECT_SAMPLE_TYPE_DRUM

detect_drum_type()

detect_drum_type(<zone-id>, <drum-type-result>)

Assigns <drum-type-result> a $NI_DETECT_DRUM_TYPE tag describing the drum type of an audio sample. You can use this function if detect_sample_type() determines that a given audio sample is of type $NI_DETECT_SAMPLE_TYPE_DRUM. If detection fails, the function will return ~NI_DETECT_DRUM_TYPE_INVALID.

<zone-id>

The ID of the zone.

<drum-type-result>

The detected drum type, can be one of the following:

$NI_DETECT_DRUM_TYPE_INVALID

$NI_DETECT_DRUM_TYPE_KICK

$NI_DETECT_DRUM_TYPE_SNARE

$NI_DETECT_DRUM_TYPE_CLOSED_HH

$NI_DETECT_DRUM_TYPE_OPEN_HH

$NI_DETECT_DRUM_TYPE_TOM

$NI_DETECT_DRUM_TYPE_CYMBAL

$NI_DETECT_DRUM_TYPE_CLAP

$NI_DETECT_DRUM_TYPE_SHAKER

$NI_DETECT_DRUM_TYPE_PERC_DRUM

$NI_DETECT_DRUM_TYPE_PERC_OTHER

detect_instrument_type()

detect_instrument_type(<zone-id>, <instr-type-result>)

Assigns <instr-type-result> a $NI_DETECT_INSTRUMENT_TYPE tag describing the instrument type of an audio sample. Hint: use this function if detect_sample_type() determines that a given audio sample is of type $NI_DETECT_SAMPLE_TYPE_INSTRUMENT. If detection fails, the function will return $NI_DETECT_INSTRUMENT_TYPE_INVALID.

<zone-id>

The ID of the zone

<instr-type-result>

The detected instrument type, can be one of the following:

$NI_DETECT_INSTRUMENT_TYPE_INVALID

$NI_DETECT_INSTRUMENT_TYPE_BASS

$NI_DETECT_INSTRUMENT_TYPE_BOWED_STRING

$NI_DETECT_INSTRUMENT_TYPE_BRASS

$NI_DETECT_INSTRUMENT_TYPE_FLUTE

$NI_DETECT_INSTRUMENT_TYPE_GUITAR

$NI_DETECT_INSTRUMENT_TYPE_KEYBOARD

$NI_DETECT_INSTRUMENT_TYPE_MALLET

$NI_DETECT_INSTRUMENT_TYPE_ORGAN

$NI_DETECT_INSTRUMENT_TYPE_PLUCKED_STRING

$NI_DETECT_INSTRUMENT_TYPE_REED

$NI_DETECT_INSTRUMENT_TYPE_SYNTH

$NI_DETECT_INSTRUMENT_TYPE_VOCAL

Examples

on init
    message("")

    declare ~pitch_result

    set_num_user_zones(1)

    declare ui_mouse_area $Drop
    declare ui_label $Info (1, 1)

    set_text($Info, "Drop sample here!")

    move_control_px($Drop, 66, 2)
    move_control_px($Info, 66, 2)

    set_control_par(get_ui_id($Drop), $CONTROL_PAR_DND_ACCEPT_AUDIO, $NI_DND_ACCEPT_ONE)
    set_control_par(get_ui_id($Drop), $CONTROL_PAR_WIDTH, 150)
    set_control_par(get_ui_id($Info), $CONTROL_PAR_WIDTH, 150)

    set_zone_par(%NI_USER_ZONE_IDS[0], $ZONE_PAR_HIGH_KEY, 127)
    set_zone_par(%NI_USER_ZONE_IDS[0], $ZONE_PAR_LOW_VELO, 1)
    set_zone_par(%NI_USER_ZONE_IDS[0], $ZONE_PAR_HIGH_VELO, 127)
    set_zone_par(%NI_USER_ZONE_IDS[0], $ZONE_PAR_GROUP, 0)
end on

on ui_control ($Drop)
    if ($NI_MOUSE_EVENT_TYPE = $NI_MOUSE_EVENT_TYPE_DROP)
        wait_async(set_sample(%NI_USER_ZONE_IDS[0], !NI_DND_ITEMS_AUDIO[0]))

        wait_async(detect_pitch(%NI_USER_ZONE_IDS[0], ~pitch_result))
        wait_async(set_zone_par(%NI_USER_ZONE_IDS[0], ...
                                $ZONE_PAR_ROOT_KEY, ...
                                int(round(~pitch_result))))
        wait_async(set_zone_par(%NI_USER_ZONE_IDS[0], ...
                                $ZONE_PAR_TUNE, ...
                                int(100.0 * (round(~pitch_result) - ~pitch_result))
    end if
end on

Set the zone root key by rounding the pitch result to an integer value, then set the zone tune to correct for the pitch offset. 

on init
    message("")

    set_num_user_zones(1)

    declare ui_mouse_area $Drop
    declare ui_label $Info (1, 1)

    declare $sample_type
    declare $instrument_type
    declare $drum_type
    declare !drum_types[10]
    !drum_types[$NI_DETECT_DRUM_TYPE_KICK]       := "Kick"
    !drum_types[$NI_DETECT_DRUM_TYPE_SNARE]      := "Snare"
    !drum_types[$NI_DETECT_DRUM_TYPE_CLOSED_HH]  := "Closed Hi-Hat"
    !drum_types[$NI_DETECT_DRUM_TYPE_OPEN_HH]    := "Open Hi-Hat"
    !drum_types[$NI_DETECT_DRUM_TYPE_TOM]        := "Tomr"
    !drum_types[$NI_DETECT_DRUM_TYPE_CYMBAL]     := "Cymbal"
    !drum_types[$NI_DETECT_DRUM_TYPE_CLAP]       := "Clap"
    !drum_types[$NI_DETECT_DRUM_TYPE_SHAKER]     := "Shaker"
    !drum_types[$NI_DETECT_DRUM_TYPE_PERC_DRUM]  := "Drum Percussion"
    !drum_types[$NI_DETECT_DRUM_TYPE_PERC_OTHER] := "Misc Percussion"
    declare !instrument_types[12]
    !instrument_types[$NI_DETECT_INSTRUMENT_TYPE_BASS]           := "Bass"
    !instrument_types[$NI_DETECT_INSTRUMENT_TYPE_BOWED_STRING]   := "Bowed String"
    !instrument_types[$NI_DETECT_INSTRUMENT_TYPE_BRASS]          := "Brass"
    !instrument_types[$NI_DETECT_INSTRUMENT_TYPE_FLUTE]          := "Flute"
    !instrument_types[$NI_DETECT_INSTRUMENT_TYPE_GUITAR]         := "Guitar"
    !instrument_types[$NI_DETECT_INSTRUMENT_TYPE_KEYBOARD]       := "Keyboard"
    !instrument_types[$NI_DETECT_INSTRUMENT_TYPE_MALLET]         := "Mallet"
    !instrument_types[$NI_DETECT_INSTRUMENT_TYPE_ORGAN]          := "Organ"
    !instrument_types[$NI_DETECT_INSTRUMENT_TYPE_PLUCKED_STRING] := "Plucked String"
    !instrument_types[$NI_DETECT_INSTRUMENT_TYPE_REED]           := "Reed"
    !instrument_types[$NI_DETECT_INSTRUMENT_TYPE_SYNTH]          := "Synth"
    !instrument_types[$NI_DETECT_INSTRUMENT_TYPE_VOCAL]          := "Vocal"

    set_text($Info, "Drop sample here!")

    move_control_px($Drop, 66, 2)
    move_control_px($Info, 66, 2)

    set_control_par(get_ui_id($Drop), $CONTROL_PAR_DND_ACCEPT_AUDIO, $NI_DND_ACCEPT_ONE)
    set_control_par(get_ui_id($Drop), $CONTROL_PAR_WIDTH, 150)
    set_control_par(get_ui_id($Info), $CONTROL_PAR_WIDTH, 150)

    set_zone_par(%NI_USER_ZONE_IDS[0], $ZONE_PAR_HIGH_KEY, 127)
    set_zone_par(%NI_USER_ZONE_IDS[0], $ZONE_PAR_LOW_VELO, 1)
    set_zone_par(%NI_USER_ZONE_IDS[0], $ZONE_PAR_HIGH_VELO, 127)
    set_zone_par(%NI_USER_ZONE_IDS[0], $ZONE_PAR_GROUP, 0)
end on

on ui_control ($Drop)
    if ($NI_MOUSE_EVENT_TYPE = $NI_MOUSE_EVENT_TYPE_DROP)
        wait_async(set_sample(%NI_USER_ZONE_IDS[0], !NI_DND_ITEMS_AUDIO[0]))
        wait_async(detect_sample_type(%NI_USER_ZONE_IDS[0], $sample_type))

        select ($sample_type)
            case $NI_DETECT_SAMPLE_TYPE_INSTRUMENT
                wait_async(detect_instrument_type(%NI_USER_ZONE_IDS[0], $instrument_type))
            case $NI_DETECT_SAMPLE_TYPE_DRUM
                wait_async(detect_drum_type(%NI_USER_ZONE_IDS[0], $drum_type))
            case $NI_DETECT_SAMPLE_TYPE_INVALID
                set_text($Info, "Cannot recognize sample type!")
        end select

        if ($sample_type = $NI_DETECT_SAMPLE_TYPE_INSTRUMENT)
            if ($instrument_type = $NI_DETECT_INSTRUMENT_TYPE_INVALID)
                set_text($Info, "Cannot recognize instrument type!")
            else
                set_text($Info, "Instrument: " & !instrument_types[$instrument_type])
            end if
        else
            if ($drum_type = $NI_DETECT_DRUM_TYPE_INVALID)
                set_text($Info, "Cannot recognize drum type!")
            else
                set_text($Info, "Instrument: " & !drum_types[$drum_type])
            end if
        end if
    end if
end on

Detect whether a sample is of type instrument or drum, and detect the corresponding drum or instrument type.