Next step for cooperation on speech-to-text technology
Links
Research cooperation to develop new German speech-to-text technology at the Martin Luther University
Interview with Professor John A Albertini, visiting professor at the Forschungsstelle zur Rehabilitation von Menschen mit kommunikativer Behinderung at the Martin Luther University Halle-Wittenberg. Professor Albertini is Chair of the Department for Research and Teacher Education at the National Technical Institute for the Deaf in Rochester (NY), where he has been teaching and conducting research since 1976. In 1999, he stayed at the Forschungsstelle zur Rehabilitation von Menschen mit kommunikativer Behinderung at the MLU as visiting professor, and has now returned for several weeks to prepare a new research project.
[Interview by Dr Christina Schröder, 17 July 2006]
Professor Albertini, you normally work at the National Technical Institute for the Deaf (NTID) in Rochester, and have just spent several weeks teaching and researching at the Martin Luther University. Why Halle?
Because of the Research Institute here in Special Education, the Forschungsstelle, led by Professor Schlenker-Schulte. I’ve been fortunate enough to have a relationship with the Forschungsstelle for now almost twenty years – first when it was in Heidelberg, and now in Halle. So, why? Because we share one, or several, common interests. My area is the acquisition of reading and writing by deaf students, and one of the interests of the Forschungsstelle – not the only of course – is the acquisition of reading and writing by students with disabilities, deafness being one of the disabilities. For me, it is a wonderful opportunity to talk to colleagues with an interest in deafness, but also to learn more about the application of my work and of common ideas to students with other disabilities. – In America, the study of deafness is separated from other areas of special education.
What does your cooperation with the Forschungsstelle consist of?
The project we have had in the past, which is still going, is something called “The Use of Dialogue Journals”. Now we are talking about a new project, which is speech-to-text technology. – Let me tell you about both. The dialogue journal is actually a teaching method, where students write informally with their instructor. It’s a one-on-one kind of writing, it’s informal, and it’s personal. But it also takes place within the context of the school, so it is also an opportunity to talk about learning and the substance of learning. But it is an informal communication. And I found with American students that it was a good way of developing an immediate, close relationship with one’s students and a good way to practice writing, to communicate openly, and – for the students – to reflect on their learning. Professor Schlenker-Schulte has also found this to be true and is now building it into her teaching with her Master students so that they are using dialogue journals in their Praktikum, in what we call their practice teaching.
So this dialogue-journals method is now being used not only with deaf students, but generally with students in order to enhance their learning process during their degrees?
Professor Schlenker-Schulte’s students have taken the method and tried it with a variety of students with other disabilities. It has been used with immigrant students and with average mainstream students, but her students are using it with deaf, with physically handicapped, and with mentally disabled students as well.
And do results suggest that all of these groups find it useful – including students who teach students?
Yes, generally, the response has been very positive. They find it is a way of establishing a meaningful communication – a way in which students can use writing to express themselves; to express their interests, their frustrations, their goals; and to practice talking about a subject. And it also seems to build the motivation for students to write. Writing is not a channel that they necessarily have had success with, but it is one they find – and the Master students find – to be actually one they can use successfully.
… and train?
Yes. For example, deaf students’ writing has always been corrected – no matter what they write. This is a kind of writing that, within the journal, is not corrected. And so they feel very free to experiment, to try different kinds of expression, and to just communicate. It works like a usual conversation: If one partner does not understand, then the other partner can ask questions.
Why do you think deaf students’ writing is corrected no matter what they write?
In the school system, it seems to be the expectation that the form be accurate, and that what is written be accurate, even if the content is not adequate. So there seems to be more weight placed on the form of communication. And with deaf students, that has always been a challenge – to learn the spoken language in its spoken form and its written form. Form, some argue, should be worked on in the schools before science or geography or the other subjects can be taught. But what has tended to happen, in my understanding, is that the focus remains on the form, and so students do not see writing as a way of communication.
Can a communication disability like deafness also involve other difficulties, such as learning difficulties, and which attempts are made to help students with such related problems?
Well, looking at the learning process of deaf students, for example, the challenge is – and the miracle is sometimes – how deaf students learn a spoken language. Now, that immediately presents communication problems, but it also presents, as you said, learning problems. If they do not have a basis in a language – it could be a spoken language or a sign language – very early, then further development in language learning will be delayed, and reading and writing will be delayed. Of course those are the two corner stones of the continued career in school and then finding work. And we are a very literate society. In the early 1900s, a printer or a farmer could learn on the job, could learn by observing, and many deaf people were very successful in that. But more and more now, a printer, for example – and I use this example here because printer was a very typical profession for deaf people – must use a computer, must be able to read instruction manuals. So the demands have increased.
What does that imply for research into deafness and related learning difficulties?
It means that, for example, a communication difficulty needs to be addressed very early and with as much care as possible. It needs to be recognized early and worked on early, and any means – like assistive communication devices – must be available to students with inability to speak or with difficulty in reasoning or with some mental developmental delay. A way must be found to encourage and to challenge that person.
And that is where the journal exercise comes back in again …
… we found that, yes. For some students, the informal writing is a very good way. And what I found with some of Professor Schlenker-Schulte’s students in Halle is that it can be drawing and painting, so that students who are severely developmentally delayed mentally can also have communication: with pictures and arrows and symbols other than words. There can be a dialogue between teacher and child in that situation as well, on a very different level, but for some who also have no ability to speak it is a break-through.
Did you find that students became more confident about their writing skills after writing those journals?
Yes, they seem to. And they become more interested in writing, more enthusiastic about writing as a tool, and more motivated to improve their writing. The focus is on what they are thinking and what they are expressing, not on the form – the form can be worked on later. That’s the principle.
Is your new project related to this research?
Not directly. This time, we’ve started talking about a new project, and that is the use of speech-to-text, or speech-to-print, technology to improve deaf and hard of hearing students’ access to instruction. It is like a written form of an interpreter, a system that either uses a technician, who can type well, understands the language well, and can learn a system of abbreviations quickly. And that allows this person to follow a lecture and to transcribe the key ideas into print. So there is some summary and some selection in the transcription from English to English, or from German to German – from spoken to written. This transcription works through software, which converts the set of abbreviations and the lexicon into print, with a delay of several seconds.
So we are basically talking about transcription in real time.
Almost, yes.
How does the transcriber learn the set of abbreviations – like another language, similar to sign language for example?
No. It’s a system that someone can learn in about forty hours. It is not word for word, so it’s not like stenography; it is more meaning to meaning. We found – in English anyway – that with about forty hours of training, continued practice and continued learning about the content of school work, there is very good accuracy and minimum delay. We are also looking into automatic speech recognition so that the technician has the choice between either typing or speaking.
… of the abbreviations?
No, this time it’s spoken German or spoken English. But: the catch is the accuracy. It is not as accurate, and the typist still needs the abbreviations to make corrections. So the typist can use either speech or typing. Both feed into the software. If the typist is relying on speech, he or she can follow it, and if there is an error, make a quick change online.
Is this programme the first of its kind that has been developed?
Yes – for educational purposes. As I said, there has been stenography for many years, but this is, to start with, less expensive then stenography; it requires less training; and it is also of interest to us as educators because it can be interactive. So, we are building new functions into the software so that the student as well as the captionist can edit the text. Normally it’s two laptops: The captionist types on one laptop, and the student in the lecture or in the class has another laptop that follows. So students can follow the instructor and also follow their laptop. But we are now working on software that will allow the student to make highlighting or bolding, make notes, and also ask the captionist questions – like a chat function.
That means that deaf students could attend “normal” lectures without any problems.
Yes, they would have access, or better access, to the lectures than some have now. For example, some who don’t know sign language need another type of access, so they would then have this printed or transcribed access.
This is being, or has been, developed in Rochester?
We have an English version, yes, that was developed by my colleague, Professor Michael Stinson, and his team. It is being used in middle schools, high schools, and in the college setting, at the college where I teach. And now we are looking into the possibility of developing a German version, a German lexicon that will use the software.
And that would happen here in Halle?
At the Forschungsstelle, we are hoping to develop a German prototype. We are just beginning discussions now, and we are looking for funding to adapt the software and a large enough lexicon to pilot the system in a classroom.
So if we’d be very optimistic – looking at this kind of progress and given the current advance in technology and computer-based solutions – could it be that in future we reach a point where there is no “communication disability” as such any more, because people affected by communication disabilities can communicate?
Well, that would be the goal, that’s an ideal. But it becomes complicated because sometimes when we create solutions for one group, we create barriers for another group. In the development of computer technology, for example, there is voice recognition built into software so that you can, if you are physically disabled, direct your computer by speech. Or, some programmes produce computer-generated speech, instead of text. Well, that is good for some, but not for deaf, people. Information depicted graphically is suitable for deaf people who rely on their vision, but not for blind people. For blind people or people with low vision, you need commentary or what they call verbally supported graphic displays. The telephone was a great step forward for hearing people, but it presented a barrier to deaf people. That barrier was overcome with the text telephone. And now, of course, we have email and instant messaging which, again, have reduced barriers in the workplace considerably. A deaf person working with hearing people can now use these electronic means of communicating.
Where do you as a senior researcher in this field see the responsibilities of society and politics in dealing with people affected by communication disabilities?
I think it is the responsibility of society to educate and to provide access to education for all its citizens, and that of course applies to people with disabilities. I think this is an ethical responsibility. And I think our responsibility as educators is to point out where there are ethical problems, barriers, inequalities, and discrepancies, and to work with others to correct those.
Do you think that those issues are better dealt with in the United States than, say, here?
No, I think the same questions and issues are being dealt with in Germany and in the US, I don’t think there are any great differences in this respect. I know Germany and the United States at least have laws now that mandate access. In the United States, for example, we have the “Americans with Disabilities Act”. That’s a law established several years ago to ensure that people with disabilities have access to education, to public facilities, and to the work place. There are similar laws in Germany.
… and with regard to research?
The same. On the one hand, we have developed the software for speech-to-print transcription to help students in classes. On the other hand, the Forschungsstelle has developed and researched the optimization of professional exams – exams that allow students to enter a particular profession. They are often written in language that is a barrier, and they have had a great deal of success with the development and the acceptance of standardized entrance tests into the professions. In their cooperation with the Rheinisch-Westfälische Berufsschule für Hörgeschädigte in Essen they are certainly ahead of us.
We have been talking mainly about the US and Germany here, but I understand that other countries also invest money into research into this field. Are findings shared on an international level, or does competition also play a role there?
Yes, there are attempts to share information. The trouble is that academics become so specialized that, sometimes, they don’t have an opportunity to see what’s going on in other areas. For this reason, I welcome this opportunity here in Halle to find out about research being conducted in other disability areas
Do you find that your kind of research is widely appreciated in society, or is it rather difficult to convince people of the urgency of your research, or of the need for funding?
Yes learning problems that are related to language learning are sometimes difficult for people to understand. Language learning is an abstraction, which is why I think student writing helps people to see the development of a person’s ability to think. I believe that helps make the abstract concrete, but it’s not like sending a man to the moon or creating a new vaccine. Those are very concrete, dramatic events that capture people’s imagination. The idea that development takes time and is not immediately apparent makes it harder to convince funders to support this kind of research.
Although the development of a new real-time speech-to-text-transcript software for deaf students is almost like sending a man to the moon, is it not?
We think so. We think it is quite exciting, quite a break-through for the students.
How “easy” do you find it to present your actual research findings in practice?
There are different ways to demonstrate progress. You have different sorts of results: quantitative and qualitative. Some policy makers only want to see quantitative data. But for language and writing development, quantitative methods are often insufficient. Depth of thought and development of thought and language are not well expressed in just quantitative terms. In America now it’s sometimes difficult to make the argument for qualitative results if the budget is the only concern, because policy makers want to see success in numbers of students. In situations where development within an individual is not easily quantified, we try to demonstrate development both ways. We design studies that use both quantitative and qualitative methods. One enriches the other, really.
So if you could make a wish, what would need to change to make your research “easier” – including aspects like funding, public awareness, public recognition, etc?
I think what you said in your question, public awareness, is a key. And that is something academicians sometimes need help with, because we are often guilty of conducting very theoretical studies where the application is not apparent. Something like the “Long Night of Research” that was recently in Halle is for me a unique experience. I don’t know of any other university – certainly not in America – that opens its doors to the city and attempts to make its work accessible to the citizens. That is something I would like my city very much to follow suit on. I think that can only have positive results. It’s not enough, but it is an essential beginning that people understand the kind of work that’s being done at the university and what possible benefits it would have for the average citizen. – So yes, something like more publications, more public relations, and more awareness is perhaps the key.