标签:sel maximum focus mini doc move 其他 log folder
语音输入是我们操作全息对象的另一种交互方式,语音指令在实际操作过程中是非常自然和容易的,设计语音指令需要考虑以下几点:
在Holograms 101的教程里,已经使用关键字识别构建了两个简单的语音指令,这节教程将更深入的学习语音输入相关的知识:
这节教程使用的项目依然是上两节教程Holograms 210 and Holograms 211完成的项目。
下载此项目所需文件 files .
语音指令的设计需要考虑一些因素,下面是一些建议:
具体发布详情参见前两节教程
这节主要实现应用可以录入用户的声音,并在记录时提供动画反馈表明正在记录
在Unity中的应用设置时 Microphone 功能必须开启 . 这个教程项目Holograms 212已经开启了麦克风功能, 但是在你自己的开发应用过程中不要忘记这一点.
// TODO: 2.a Delete the following two lines: RecordButton.SetActive(false); MessageUIRenderer.gameObject.SetActive(false);
这一章主要使用听写识别器来将用户录入的声音转换为文字,并显示在通信面板上。
使用听写识别器(Dictation Recognizer)需要考虑到以下几点:
1 必须使Hololens连接到WiFi才能使听写识别器工作。
2 超时发生在设定的时间段之后。 有两个超时要注意:
3 如果识别器启动,并且在前五秒没有听到任何音频,它将超时。
4 如果识别器听写过程中,听到了二十秒的静音,它将超时。
5 一次只能运行一种类型的识别器(关键字识别器或听写识别器)。
同样这个功能也需要Unity的项目设置中开启麦克风功能。在您个人的开发项目中不要忘记这一点。
接下来需要重新编辑MicrophoneManager.cs脚本,完成以下几点:
using HoloToolkit; using System.Collections; using System.Text; using UnityEngine; using UnityEngine.UI; using UnityEngine.Windows.Speech; public class MicrophoneManager : MonoBehaviour { [Tooltip("A text area for the recognizer to display the recognized strings.")] public Text DictationDisplay; private DictationRecognizer dictationRecognizer; // Use this string to cache the text currently displayed in the text box. private StringBuilder textSoFar; // Using an empty string specifies the default microphone. private static string deviceName = string.Empty; private int samplingRate; private const int messageLength = 10; // Use this to reset the UI once the Microphone is done recording after it was started. private bool hasRecordingStarted; void Awake() { /* TODO: DEVELOPER CODING EXERCISE 3.a */ // 3.a: Create a new DictationRecognizer and assign it to dictationRecognizer variable. dictationRecognizer = new DictationRecognizer(); // 3.a: Register for dictationRecognizer.DictationHypothesis and implement DictationHypothesis below // This event is fired while the user is talking. As the recognizer listens, it provides text of what it‘s heard so far. dictationRecognizer.DictationHypothesis += DictationRecognizer_DictationHypothesis; // 3.a: Register for dictationRecognizer.DictationResult and implement DictationResult below // This event is fired after the user pauses, typically at the end of a sentence. The full recognized string is returned here. dictationRecognizer.DictationResult += DictationRecognizer_DictationResult; // 3.a: Register for dictationRecognizer.DictationComplete and implement DictationComplete below // This event is fired when the recognizer stops, whether from Stop() being called, a timeout occurring, or some other error. dictationRecognizer.DictationComplete += DictationRecognizer_DictationComplete; // 3.a: Register for dictationRecognizer.DictationError and implement DictationError below // This event is fired when an error occurs. dictationRecognizer.DictationError += DictationRecognizer_DictationError; // Query the maximum frequency of the default microphone. Use ‘unused‘ to ignore the minimum frequency. int unused; Microphone.GetDeviceCaps(deviceName, out unused, out samplingRate); // Use this string to cache the text currently displayed in the text box. textSoFar = new StringBuilder(); // Use this to reset the UI once the Microphone is done recording after it was started. hasRecordingStarted = false; } void Update() { // 3.a: Add condition to check if dictationRecognizer.Status is Running if (hasRecordingStarted && !Microphone.IsRecording(deviceName) && dictationRecognizer.Status == SpeechSystemStatus.Running) { // Reset the flag now that we‘re cleaning up the UI. hasRecordingStarted = false; // This acts like pressing the Stop button and sends the message to the Communicator. // If the microphone stops as a result of timing out, make sure to manually stop the dictation recognizer. // Look at the StopRecording function. SendMessage("RecordStop"); } } /// <summary> /// Turns on the dictation recognizer and begins recording audio from the default microphone. /// </summary> /// <returns>The audio clip recorded from the microphone.</returns> public AudioClip StartRecording() { // 3.a Shutdown the PhraseRecognitionSystem. This controls the KeywordRecognizers PhraseRecognitionSystem.Shutdown(); // 3.a: Start dictationRecognizer dictationRecognizer.Start(); // 3.a Uncomment this line DictationDisplay.text = "Dictation is starting. It may take time to display your text the first time, but begin speaking now..."; // Set the flag that we‘ve started recording. hasRecordingStarted = true; // Start recording from the microphone for 10 seconds. return Microphone.Start(deviceName, false, messageLength, samplingRate); } /// <summary> /// Ends the recording session. /// </summary> public void StopRecording() { // 3.a: Check if dictationRecognizer.Status is Running and stop it if so if (dictationRecognizer.Status == SpeechSystemStatus.Running) { dictationRecognizer.Stop(); } Microphone.End(deviceName); } /// <summary> /// This event is fired while the user is talking. As the recognizer listens, it provides text of what it‘s heard so far. /// </summary> /// <param name="text">The currently hypothesized recognition.</param> private void DictationRecognizer_DictationHypothesis(string text) { // 3.a: Set DictationDisplay text to be textSoFar and new hypothesized text // We don‘t want to append to textSoFar yet, because the hypothesis may have changed on the next event DictationDisplay.text = textSoFar.ToString() + " " + text + "..."; } /// <summary> /// This event is fired after the user pauses, typically at the end of a sentence. The full recognized string is returned here. /// </summary> /// <param name="text">The text that was heard by the recognizer.</param> /// <param name="confidence">A representation of how confident (rejected, low, medium, high) the recognizer is of this recognition.</param> private void DictationRecognizer_DictationResult(string text, ConfidenceLevel confidence) { // 3.a: Append textSoFar with latest text textSoFar.Append(text + ". "); // 3.a: Set DictationDisplay text to be textSoFar DictationDisplay.text = textSoFar.ToString(); } /// <summary> /// This event is fired when the recognizer stops, whether from Stop() being called, a timeout occurring, or some other error. /// Typically, this will simply return "Complete". In this case, we check to see if the recognizer timed out. /// </summary> /// <param name="cause">An enumerated reason for the session completing.</param> private void DictationRecognizer_DictationComplete(DictationCompletionCause cause) { // If Timeout occurs, the user has been silent for too long. // With dictation, the default timeout after a recognition is 20 seconds. // The default timeout with initial silence is 5 seconds. if (cause == DictationCompletionCause.TimeoutExceeded) { Microphone.End(deviceName); DictationDisplay.text = "Dictation has timed out. Please press the record button again."; SendMessage("ResetAfterTimeout"); } } /// <summary> /// This event is fired when an error occurs. /// </summary> /// <param name="error">The string representation of the error reason.</param> /// <param name="hresult">The int representation of the hresult.</param> private void DictationRecognizer_DictationError(string error, int hresult) { // 3.a: Set DictationDisplay text to be the error string DictationDisplay.text = error + "\nHRESULT: " + hresult; } private IEnumerator RestartSpeechSystem(KeywordManager keywordToStart) { while (dictationRecognizer != null && dictationRecognizer.Status == SpeechSystemStatus.Running) { yield return null; } keywordToStart.StartKeywordRecognizer(); } }
这章将使用Grammar Recognizer来识别用户的语音,这需要借鉴SRGS(语音识别语法规范)文件。
同样这个功能也需要Unity的项目设置中开启麦克风功能。在您个人的开发项目中不要忘记这一点。
1 在Unity中重建应用程序,然后从Visual Studio构建和部署以在HoloLens上体验应用程序。
2 凝视宇航员的喷气背包,并执行air tap手势。
3 开始说话。 语法识别器将解释您的语音,并根据识别结果更改形状的颜色。 示例命令是“blue circle, yellow square”。
4 再一次使用air tap手势以关闭工具箱。
原文地址:https://developer.microsoft.com/en-us/windows/holographic/holograms_212
如有翻译上的理解与错误请指正,谢谢哦!
微软Hololens学院教程-Hologram 212-Voice(语音)
标签:sel maximum focus mini doc move 其他 log folder
原文地址:http://www.cnblogs.com/qichun/p/6059867.html