ARTTS - Assistive Real-Time Technology in Singing

  • Principal Investigator: Aníbal João de Sousa Ferreira | CINTESIS/FMUP contact:  Este endereço de email está protegido contra piratas. Necessita ativar o JavaScript para o visualizar.
  • Researchers
    • Ana Sofia Almeida de Sá Serra Dawa
    • Altamiro Manuel Rodrigues da Costa Pereira
    • Manuel António Caldeira Pais Clemente
    • Daniela da Costa Coimbra
    • Álvaro Manuel Mendes Barbosa
    • Sandra de Oliveira Dias
    • Nuno Miguel da Costa Santos Fonseca
    • Susana Alexandre Vaz dos Santos Freitas
    • Rui Luís dos Reis Faria Taveira
    • Vítor Filipe Maia Baptista Fonseca Almeida
  • Fellows
    • Tiago José da Silva Campos
    • Inês Catarina Silva Pereira de Moura
  • Start date: 01.10.2010 | End date: 30.09.2013
  • Financing: € 190,000.00 (FCT)

Project description: Singing is perhaps the most expressive and effective modality for humans to communicate emotions, feelings, ideas, knowledge and artistic attitude. Singing is endogenous to every culture or society and it has always played a key role on the way cultural identity is shaped and transmitted across generations.
While in practice all human individuals develop an appreciation for music and singing, a significant fraction of the world population is strongly committed to educate its singing skills either motivated by personal or professional artistic realization.

It is quite a paradox however that on the transition between the XX and XXI centuries, a period in technological history known as "information age”, characterized by the ubiquity of the computer and inspired by the concept of "ambience intelligence"; the pedagogy of singing, the assessment of the quality of singing and the preventive vocal usage are poorly assisted by computers. For example, it is quite revealing that while Wakefield discusses the role of subjectivity in the communication and appreciation of singing, and concludes that the proprioceptive feedback is insufficient [Wak03], a study on the use in singing training of real-time visual feedback technology shows that it has a clear positive impact in improving singing abilities [HopO6].

This project proposal addresses those issues in an ambitious way by gathering together institutions, professionals and researchers from three complementary areas: singing pedagogy, engineering/signal processing, and medical/Laryngology. The common purpose is to articulate knowledge and know-how from the different disciplines in order to design, implement and validate innovative technologies and methodologies that are useful to singing students, teachers or professionals, namely:

  1. new technology-assisted pedagogic methodologies,
  2. real-time visual feedback of relevant quality parameters of the singing voice, and
  3. real-time monitoring and assessment of the singing voice with the purpose to prevent voice disorders.

In order to address these challenges seven tasks have been planned that include the following specific goals:

  • to promote a deep and structured knowledge concerning the voice production system, the correlation between subjective quality parameters of the singing voice (e.g., breathiness, clarity, vibrato, singer's formant) and objective acoustic features (e.g., jitter, shimmer, harmonics to noise ratio, harmonic irregularity and extension, closing/open coefficient of the glottal pulse), the correlation between objective acoustic features and voice disorders in singing,
  • the design, realization and validation of biofeedback technologies in singing as well as technology-assisted teaching/learning methodologies,
  • the design, realization and optimization of technologies allowing the transcription of singing to musical score and including editing capabilities,
  • the robust estimation of the glottal pulse in real-time from running singing and not only from sustained vowels as it is the rule with currently existing technology with the purpose to extract information concerning the quality of the phonation or the abnormal operation of the vocal folds,
  • the design, realization and validation of technologies for the real-time assessment of the singing voice in order to monitor vocal stress, to detect risks of voice over-use and to prevent voice disorders.

The expertise and professional experience of the team that has been gathered around this project proposal not only covers the different axis of the problem, namely singing pedagogy, engineering and laryngology, but also contributes with quite relevant research results from other research projects or on-going PhD work. For example the Principal Investigator (PI) has conducted or coordinated research for more than 20 years, notably in the areas of audio coding/compression [Fer99, Fer08], acoustic signal processing [Fer06] and accurate voice/audio interpretation and transcription [Fer01, Fer02, Fer07, Fer08_2]. The PI has also been the principal promoter between 2004 and 2008 of a spin-off company in the areas of acoustic vocal assessment, biofeedback in stuttering treatment and visual feedback of the melodic line in singing. Currently the PI supervises three PhD students in the areas of new acoustic features in speech and singing, correlation between acoustic features and voice perceptual parameters, and singing to synthetic voice transcription. This team will contribute significantly to the project, namely with databases of normal and pathological voices and of singing. Project partners from the Faculty of Medicine of the University of Porto, from the School of Arts, Sound and Image of the Portuguese Catholic University and from the Performing Arts College of the Polytechnic Institute of Porto bring invaluable expertise to the project respectively in the areas of laryngology and vocal assessment (e.g.,   interactive systems in music, and singing teaching practice and methodologies.

Secção Ensino Secção investigacao Secção consultadoria
   Mestrados | Doutoramentos | Formação Contínua         Projetos | Teses e Dissertações | Publicações        Bioestatística | Informática Médica