Workpackage WP1:
AUDIO FEATURE EXTRACTION
General objectives:
- Construction of a digital library of musical audio data of interest to the project.
- Development of the necessary components for the low and high-level description of music and sound.
- Development of a set of tools for the capture of gestures during musical performances.
- Development of a library for content-based transformation of music and sound.
Task T1.1: Corpus generation and acquisition.
Objectives: To record, acquire and organize musical audio data of interest
to the project.
Task T1.2: Development of components for the low-level description of musical content.
Objectives: To develop a library of music and sound low-level description components appropriate to the problems tackled in the project.
Task T1.3: Development of components for the high-level semantic description of musical content.
Objectives: To develop a library of music and sound high-level semantic description components appropriate to the problems tackled in the project.
Task T1.4: Development of components for musical gesture feature extraction.
Objectives: To develop a set of tools for capturing of gestures during musical performances.
Task T1.5: Study of physical transformation of music and sound components.
Objectives: To accurately determine the required functionalities for the physical transformation of music and sound components.
Subtask T1.6: Development of physical transformation of music and sound components.
Objectives: To develop a library of content-based transformation components for musical recordings.
Workpackage WP2:
SYMBOLIC MUSIC FEATURE EXTRACTION
General objectives:
- Compilation of the necessary corpora for the tasks addressed.
- Development of a rule-based system for melody track identification in multitrack symbolic music files.
- Definition of a set of statistical and linguistic descriptors for monophonic, polyphonic and rhythmic sequences.
- Development of a system for tonal analysis able to describe in a human-readable language the decisions taken.
- Development of a system for the identification of variations on the same theme.
Task T2.1: GENERATION AND COMPILATION OF DATABASES.
Objectives: Acquisition, description and organization of data in different symbolic formats to build corpora for the experiments.
Task T2.2: MELODY TRACK IDENTIFICATION, SEGMENTATION AND TRACKING.
Objectives: To develop efficient and human-readable rules for melody track identification in multitrack symbolic music files. In the first phase, a
single decision can be enough but in the long run, a system able to swap
among different tracks tracking the melody if changes occur should be
desirable. The algorithm should be flexible enough to be adapted to any
other category of music part.
Task T2.3: MELODIC DESCRIPTORS.
Objective: To produce a set of statistical and linguistic descriptors of monophonic
sequences of notes. They must focus on the horizontal dimension of music, so we will
refer them as 'horizontal models'.
Task T2.4: HARMONIC DESCRIPTORS.
Objective: To produce a set of statistical and linguistic descriptors of
polyphonic sequences of notes. They must focus on the vertical dimension of
music, on how notes are structured in chords, so we will refer them as
'vertical models'.
Task T2.5: RHYTHMIC DESCRIPTORS.
Objective: To produce a set of statistical and linguistic rhythm descriptors of MIDI files. They will include timing, meter structure, and rhythm patterns.
Task T2.6: TONAL ANALYSIS.
Objectives: to perform a tonal analysis of the input work extracting the
melodic type of notes, tonalities, chords with their degree and tonal
functions, cadences, and modulations. Our objective is not only to obtain a
high percentage of correct analyses, but also to describe in a
human-readable way the reasons why the system has chosen an analysis as the
best one.
Task T2.7: MELODIC REDUCTION.
Objectives: to build a system able to detect and remove the elaborations or
ornamentations of a melody such that applied to two different variations of
the same theme yields again that original theme.
Task T2.8: POLYPHONIC SIMILARITY.
Objectives: to compare polyphonic music in symbolic format in order to
identify variations or covers on the same theme.
Workpackage WP3:
PROTOTYPE DEVELOPMENT
General objectives:
to integrate the results of the research from WP1 and
WP2 in prototypes for music mining, study, personalization, and information
retrieval. The following prototypes are planned to be developed:
- An interactive polyphonic music transcriptor, aimed at minimizing the number of corrections needed to get a correct solution.
- An interactive tonal music analyser conceived as an educational tool, designed according to the needs of a music school.
- An expressive music performance system for sound post-production.
- An automatic performer identification system for music information retrieval and recommendation.
- A genre classification system with a graphical interface for MIR.
- A helper tool for the handling of musical recordings in the context of sound post-production.
Task T3.1: THEORETICAL BACKGROUND.
Objective: To study the properties of interactive models able to solve
sequential solutions, improving its performance and the subjacent models
from user feedback. To analyze the problem of integrating multimodal data
from different data sources (audio, symbols, users) and different
description models (horizontal, vertical and rhythm) into single decisions.
To explore evaluation methods adapted to the interactive nature of some of
the proposed prototypes.
Task T3.2: INTERACTIVE POLYPHONIC MUSIC TRANSCRIPTION.
Objectives: To implement a prototype of an interactive polyphonic music
transcriptor. The system must provide an initial transcription of a digital
audio file containing polyphonic music and permit a human expert to make
corrections on that transcription in order to get a final, correct,
solution. The objective will be to minimize the number of corrections needed
and improve the model with them.
Task T3.4: EXPRESSIVE MUSIC PERFORMANCE PROTOTYPE.
Objective: Development of the expressive music performance system prototype in the context of sound post-production.
Task T3.5 AUTOMATIC PERFORMER IDENTIFICATION.
Objective: Development of an automatic performer identification system prototype in the context of music information retrieval and recommendation.
Task T3.6: MUSIC GENRE CLASSIFICATION.
Objective: Development of a graphic interface for music genre
classification. The system must be able to deal with both symbols and audio,
integrating all the different descriptions computed through the methods
developed in tasks T1.2, T1.3, T2.3, T2.4, and T2.5 under a multimodal
approach.
Task T3.7 INTERNET-BASED AUDIO SEARCH AND TRANSFORMATION.
Objective: Development of an Internet-based framework for audio recordings sharing, content-based search and transformation.
Workpackage WP4:
MONITORING AND EVALUATION
General objectives:
monitoring, control and evaluation of the different
stages of the project. Adaptation of the working plan to possible
eventualities. Coordination of diffusion and exploitation activities.
Task T4.1: EVALUATION AND COORDINATION MEETINGS.
Objectives: monitoring, control and evaluation of the different stages of the project.
Task T4.2: COMMUNICATION CHANNELS: WEBSITE.
Objectives: Provide an easy and fast communication and data interchange between both groups. Facilitate dissemination and exploitation of the project results. Provide visibility to the project.
Task T4.3: PUBLICATIONS AND CONFERENCES.
The participating research groups plan to organize international meetings
(i.e. international workshops and conferences) in the project area, as a
continuation of the events they have organized in the past, like the
International Workshop on Music and Artificial Intelligence, co-located with
IJCAI 2007 (Hyderabad, India), and the International Workshop on Machine
Learning and Music, co-located with ICML 2008 (Helsinki, Finland).
