TIMuL is an I+D+i project of The Spanish Ministry of Economy and Competitiveness

TIMuL is a research project of the University of Alicante and the University Pomeu Fabra of Barcelona, funded by the Ministry of economy and competitiveness, oriented to the study and development of algorithms and systems that can help to improve the process of learning and performance of music, from a pedagogical and scientific perspective. This project aims to investigate and explore the relevant aspects to develop methods and tools for music education, taking into account factors such as expressiveness, the human-computer interaction, control of movements, etc.

TIMuL es un proyecto de investigació coordinado entre la Universidad de Alicante y la Universitat Pompeu Fabra de Barcelona financiado por el Ministerio de Economía y Competitividad orientado al estudio y desarrollo de algoritmos y sistemas que puedan ayudar a mejorar el proceso de aprendizaje e interpretación de la música, desde una perspectiva pedagógica y científica. Este proyecto tiene como objetivo investigar y explorar los aspectos relevantes para elaborar métodos y herramientas para la educación musical, teniendo en cuenta factores clave como la expresividad, la interacción hombre-máquina, el control de los gestos, etc.

Worklines

The main scientific and technological objectives of the project are the following:
  • to acquire and evaluate multimodal data for the pattern recognition and machine learning techniques involved. These data include audio, gesture, physiological, symbolic, image, and video information.
  • to investigate, compare, and design techniques for integrating the descriptions coming from the different information sources involved in multimodal problems, both at feature level and decision level. Also, interactive and adaptive methodologies will be studied and developed in order to take advantage of the interaction with users or their environment.
  • Implement novel multi-modal interactive music learning prototypes compatible with traditional teaching methods for different learning tasks, based on audio, image and motion acquisition, and augmented feedback systems. Five interactive multimodal prototypes will be developed for pedagogical music learning and training:
    • Music transcription will help musicians to get scores from audio.
    • Interactive music analysis is useful get chordal and tonal information from scores for academic analysis.
    • A performances database will be developed to help musicians to learn how to play musical instruments.
    • Optical music recognition helps musicians to write music notation by hand, helping musicians to correctly write the notes.
  • Pedagogical implementation and validation of the results using novel techniques for human assessment, incorporation and evaluation of the benefits of collaborative learning in a student's network environment in close collaboration with the involved institutions. The project will develop a student network platform in which performances can be uploaded, downloaded, examined and commented, and the benefits of collaborative learning in this setting will be assessed.

Publications

Calvo-Zaragoza, J.; Oncina, J. (2017)
"An efficient approach for Interactive Sequential Pattern Recognition". Pattern Recognition, vol. 64, pp. 295-304.
Calvo-Zaragoza, J.; Oncina, J. (2017)
"Recognition of Pen-based Music Notation with Finite-State Machines". Expert Systems With Applications, vol. 72, pp. 395-406.
Rizo, D.; Marsden, A. (2016)
"A standard format proposal for hierarchical analyses and representations". Proceedings of the 3rd International Workshop on Digital Libraries for Musicology, ISBN: 978-1-4503-4751-8, pp. 25--32, New York, USA.
Valero-Mas, J.J., Calvo-Zaragoza, J. Rico-Juan, J.R. Iñesta (2016)
"An Experimental Study on Rank Methods for Prototype Selection". Soft Computing.
Plácido R. Illescas (2016)
"Análisis tonal asistido por ordenador". Universidad de Alicante, Alicante.
Valero-Mas, J.J.; Benetos, E.; Iñesta, J.M. (2016)
"Classification-based Note Tracking for Automatic Music Transcription". Proceedings of the 9th Machine Learning and Music Workshop (MML2016), pp. 61--65, Riva del Garda, Italy.
Calvo-Zaragoza, J.; Oncina, J.; De la Higuera, C. (2016)
"Computing the Expected Edit Distance from a String to a PFA". 21st International Conference Implementation and Application of Automata, pp. 39-50.
Ponce de León, P.J., Iñesta, J.M., Calvo-Zaragoza, J., Rizo, D. (2016)
"Data-based melody generation through multi-objective evolutionary computation". Journal of Mathematics and Music, vol. 10, pp. 173-192.
Calvo-Zaragoza, J.; Toselli, A. H.; Vidal, E. (2016)
"Early Handwritten Music Recognition with Hidden Markov Models" 15th International Conference on Frontiers in Handwriting Recognition.
Bernabeu, M.; Pertusa, A.; Gallego, A.J. (2016)
"Image spatial verification using Segment Intersection of Interest Points" Proc. of the 24 Int. Conf. in Central Europe on Computer Graphics, Visualization and Computer Vision (WSCG), ISBN: 2464-4614.
Rizo, D; Illescas, P.R; Iñesta, J.M. (2016)
"Interactive melodic analysis". Computational Music Analysis, ISBN: 978-3-319-25931-4, pp. 191-219.
Bellet, A; Bernabeu, J.F.; Habrard, A.; Sebban, M (2016)
"Learning discriminative tree edit similarities for linear classification - Application to melody recognition". Neurocomputing, vol. 214, pp. 155-161.
Calvo-Zaragoza, J.; Micó, L.; Oncina, J. (2016)
"Music staff removal with supervised pixel classification". International Journal on Document Analysis and Recognition, vol. 19, pp. 211-219.
Valero-Mas, J.J.; Calvo-Zaragoza, J.; Rico-Juan, J.R. (2016)
"On the suitability of Prototype Selection methods for kNN classification with distributed data". Neurocomputing, vol. 203, pp. 150-160.
Calvo-Zaragoza, J.; Valero-Mas, J.J.; Rico-Juan, J.R. (2016)
"Prototype Generation on Structural Data using Dissimilarity Space Representation". Neural Computing and Applications.
Calvo-Zaragoza, J.; Valero-Mas, J.J.; Rico-Juan, J.R. (2016)
"Selecting promising classes from generated data for an efficient multi-class NN classification". Soft Computing.
Bosch, V.; Calvo-Zaragoza, J.; Toselli, A. H.; Vidal, E. (2016)
"Sheet Music Statistical Layout Analysis". 15th International Conference on Frontiers in Handwriting Recognition.
Calvo-Zaragoza, J.; Rizo, D.; Iñesta, J.M. (2016)
"Two (note) heads are better than one: pen-based multimodal interaction with music scores". 17th International Society for Music Information Retrieval Conference, ISBN: 978-0-692-75506-8, pp. 509-514, New York City.
Rafael Ramirez, Sergio Giraldo and Zacharias Vamvakousis (2015)
"A Brain-Computer Expressive Music Performance Interface". CMMR2015: International Workshop on BCMI.
Rizo, D.; Iñesta, J.M (2015)
"A grammar for Plaine and Easie Code". Proceedings of the Music Encoding Initiative Conferences 2013 and 2014, pp. 54--64.
Valero-Mas, J.J.; Salamon, J.; Gómez, E. (2015)
"Analyzing the influence of pitch quantization and note segmentation on singing voice alignment in the context of audio-based Query-by-Humming". Proceedings of the 12th Sound and Music Computing Conference (SMC), pp. 371--378, Maynooth, Ireland.
Vamvakousis, Z., & Ramirez R. (2015)
"A PERCUSSIVE MUSICAL INTERFACE FOR A QUADRIPLEGIC PATIENT". International Symposium on Performance Science 2015. 60-61.
Bantula, H., Giraldo, S. & Ramirez R. (2015)
"A Rule Based System to Transcribe Guitar Melodies". CMMR2015: International Symposium on Computer Music Multidisciplinary Research.
Calvo-Zaragoza, J.; Barbancho, I; Tardón, L. J. ; Barbancho A. M. (2015)
"Avoiding staff removal stage in optical music recognition: application to scores written in white mensural notation". Pattern Analysis and Applications, vol. 18, pp. 933-943.
Calvo-Zaragoza, J.; Oncina, J. (2015)
"Clustering of Strokes from Pen-based Music Notation: An Experimental Study". 7th Iberian Conference on Pattern Recognition and Image Analysis (IbPRIA), ISBN: 978-3-319-19389-2, pp. 633-640, Santiago de Compostela, Spain.
Giraldo, S. & Ramirez, R. (2015)
"Computational Generation and Synthesis of Jazz Guitar Ornaments using Machine Learning Modeling. International Workshop on Machine Learning and Music (MML 2015).
Giraldo, S. & Ramirez, R. (2015)
"Computational Modeling and Synthesis of Timing, Dynamics and Ornamentation in Jazz Guitar Music". International Symposium on Computer Music Multidisciplinary Research (CMMR 2015).
Giraldo, S. & Ramirez, R. (2015)
"Computational Modeling of Ornamentation in Jazz Guitar Music". International Symposium in Performance Science (ISPS 2015).
Rafael Ramirez and Karen Lacey (2015)
"EEG-BASED CLASSIFICATION OF MUSIC-INDUCED EMOTIONS IN DANCERS: A MACHINE LEARNING APPROACH". 8th International Workshop on Machine Learning and Music. 55-56.
Vamvakousis, Z., & Ramirez R. (2015)
"EEG Signal classification in a Brain-Computer Music Interface". 8th International Workshop on Machine Learning and Music. 28-30.
J.M. Iñesta, P.J. Ponce de León, J. Calvo-Zaragoza, and D. Rizo (2015)
"Genre-based melody generation through multi-objective genetic algorithms". Proceedings of the 8th Machine Learning and Music workshop (MML 2015), Vancouver (Canada).
Rico-Juan, J. R; Calvo-Zaragoza, Jorge (2015)
"Improving classification using a Confidence Matrix based on weak classifiers applied to OCR". Neurocomputing, vol. 151, pp. 1354-1361.
Calvo-Zaragoza, J; Valero-Mas, J. J.; Rico-Juan, J. R (2015)
"Improving kNN multi-label classification in Prototype Selection scenarios using class proposals". Pattern Recognition, vol. 48, pp. 1608-1622.
Valero-Mas, J.J.; Iñesta, J.M.
"Interactive onset detection in audio recordings". Late Breaking/Demo extended abstract, 16th International Society for Music Information Retrieval Conference (ISMIR), Málaga, Spain.
Vamvakousis, Z., & Ramirez R. (2015)
"Is an auditory P300-based Brain-Computer Musical Interface feasible?". CMMR2015: International Workshop on BCMI.
Rafael Ramirez, Manel Palencia, Sergio Giraldo, Zacharias Vamvakousis (2015)
"Musical Neurofeedback for Treating Depression in Elderly People". Frontiers in Neuroscience.
Giraldo, S. & Ramirez, R. (2015)
"Performance to Score Sequence Matching for Automatic Ornament Detection in Jazz Music". ICNMC 2015: International Conference on New Music Concepts.
Calvo-Zaragoza, J.; Valero-Mas, J.J.; Rico-Juan, J.R. (2015)
"Prototype Generation on Structural Data using Dissimilarity Space Representation: A Case of Study". 7th Iberian Conference on Pattern Recognition and Image Analysis (IbPRIA), ISBN: 978-3-319-19389-2, pp. 72-82, Santiago de Compostela, Spain.
Rafael Ramirez and Sergio Giraldo (2015)
"TECHNOLOGY-ENHANCED EXPRESSIVE MUSIC PERFORMANCE LEARNING: A MACHINE LEARNING-BASED TUTORING TOOL". 8th International Workshop on Machine Learning and Music. 143-144.
Micó, L.; Sanches, J.; Cardoso, J. (2015)
"The vitality of pattern recognition and image analysis". Neurocomputing, vol. 150, pp. 124-125.
Bantulá, H., Giraldo, S. & Ramirez, R. (2014)
"A Rule Based System to Transcribe Guitar Melodies". International Workshop on Machine Learning and Music MML 2014.
Micó, L.; Oncina, J. (2014)
"Dynamic Insertions in TLAESA fast NN Search Algorithm". Proceedings of the 22nd International Conference on Pattern Recognition, ICPR, ISBN: 978-1-4799-5208-3, Stockholm, Sweden.
Illescas, P.R.; Rizo, D.; Iñesta, J.M. (2014)
"Melodic analysis of polyphonic music using an interactive pattern recognition tool". Proc. of 7th Machine Learning and Music (MML2014), Barcelona.
Valero-Mas, J.J.; Iñesta, J.M.; Pérez-Sancho, C. (2014)
"Onset detection with the user in the learning loop". Proceedings of the 7th International Workshop on Music and Machine Learning (MML2014), Barcelona, Spain.
Giraldo, S. & Ramirez, R. (2014)
"Optimizing Melodic Extraction Algorithm For Jazz Guitar Recordings Using Genetic Algorithms". International Computer Music Conference/Sound and Music Computing Conference.
Vamvakousis, Z., & Ramirez R. (2014)
"P300 Harmonies: A Brain-Computer Musical Interface". International Computer Music Conference/Sound and Music Computing Conference.
Calvo-Zaragoza, J.; Oncina, J. (2014)
"Recognition of Pen-based Music Notation with Probabilistic Machines". Proceedings of the 7th International Workshop on Machine Learning and Music , Barcelona, Spain.
Marchini, M., Ramirez R., Papiotis P., & Maestre E. (2014)
"The Sense of Ensemble: a Machine Learning Approach to Expressive Performance Modelling in String Quartets". Journal of New Music Research. 43, 303-317.


People

gRFIA-UA group

+info
Equipo Investigador:

José Manuel Iñesta Quereda (Contact)
Jose Oncina Carratalá
María Luisa Micó Andrés
Juan Ramón Rico Juan
Jorge Calera Rubio
Carlos Pérez Sancho
Pedro Ponce de León Amador
Antonio Pertusa Ibañez

Equipo de trabajo:

Javier Gallego-Sánchez
David Rizo Valero
Placido Illescas Casanova
María Hontanilla Alfonso
Jose Francisco Bernabeu Briones
Maria Luisa Bernabeu Lledó
Aureo Serrano Díaz-Carrasco
José Javier Valero Mas
Jorge Calvo Zaragoza
Javier Sober Mira

MTG-UPF group

+info
Equipo Investigador:

Rafael Ramírez-Meléndez (Contact)
Melissa Mercadal Coll
Dolors Sala Batlle
Manel Palencia-Lefler
Equipo de Trabajo:
Esteban Maestre
Alfonso Antonio Pérez Carrillo
Maria Cristina Marinescu
Oscar Mayor Soto
Panagiotis Papiotis
Sergio Iván Giraldo Méndez
Zacharias Vamvakousis
Alvaro Sarasúa Berodia

Downloads

Software and Databases developed in the project.


A prototype for genetic melody composition
  • Description: a Java software provided with a graphical interface for generating melodies using genetic algorithms based on trees. The system needs monophonic (single track) MIDI files for training. If you need some data, those used for the publication below can be requested to the authors.
  • Version: under development
  • Date: 22/02/2016
  • References: P.J. Ponce de León, J.M. Iñesta, J. Calvo-Zaragoza, and D. Rizo (2016). "Data-based melody generation through multi-objective evolutionary computation" Journal of Music and Mathematics (to be published)
  • Download (Developed in Java Version: 1.8 u77)
  • Requirements:
    - Java SE Runtime Environment is needed. You can download here.
  • User Interface.