技术领域technical field
本发明涉及在线教育移动终端设计领域，尤其涉及一种基于Cordova的跨平台在线教育移动终端设计方法。The invention relates to the field of online education mobile terminal design, in particular to a Cordova-based cross-platform online education mobile terminal design method.
背景技术Background technique
近年来，国家在宏观政策层面对教育行业予以高度重视，云计算等相关技术被在线教育行业广泛应用和推广，终身学习的理念激发人们对知识和技能的渴求，在众多因素的作用下，在线教育市场迎来了爆发式的增长。In recent years, the state has attached great importance to the education industry at the macro policy level. Cloud computing and other related technologies have been widely used and promoted in the online education industry. The concept of lifelong learning stimulates people's thirst for knowledge and skills. The education market has ushered in explosive growth.
根据中国互联网络信息中心(CNNIC)发布的《第40次中国互联网络发展状况统计报告》，截至2017年6月，我国在线教育用户规模达1.44亿，较2016年底增加662万人；手机在线教育用户规模为1.20亿，与2016年底相比增长2192万人，增长率为22.4％。移动教育正逐步成为在线教育的主流，CNNIC调查数据显示，83.3％的在线教育用户分布在手机端。与PC端相比，移动教育能提供个性化的学习场景，借助移动设备的触感、语音输出等方式，构建出更加个性化的人机交互场景，提升学习本身的趣味性，尤其对于题库类、数字阅读类、音频类在线教育产品，更适合从移动端切入。长远来看，基于移动终端，拥有优质教学内容、能寓教于乐的教育产品，在市场上更有优势。According to the "40th Statistical Report on Internet Development in China" issued by China Internet Network Information Center (CNNIC), as of June 2017, the number of online education users in my country reached 144 million, an increase of 6.62 million compared with the end of 2016; mobile online education The number of users is 120 million, an increase of 21.92 million or 22.4% compared with the end of 2016. Mobile education is gradually becoming the mainstream of online education. CNNIC survey data shows that 83.3% of online education users are distributed on mobile phones. Compared with the PC side, mobile education can provide personalized learning scenarios. With the help of the touch and voice output of mobile devices, a more personalized human-computer interaction scenario can be built to improve the fun of learning itself, especially for question banks, Digital reading and audio online education products are more suitable to start from the mobile terminal. In the long run, based on mobile terminals, education products with high-quality teaching content and entertaining education will have more advantages in the market.
作为学习教育类app的杀手锏，拍照搜题功能的强弱是一款软件好用与否的重要评判标准。互联网+教育的真正魅力就在于能够无视地域、距离、时间等客观因素，使学生能够通过app进行自主学习。在线教育平台的相互交流学习功能也是一个重点，学习者能通过app找到与自己相同水平的学习者相互交流学习和竞技。同样，功能再强大，如果没有一个美丽的界面也很难得到同学们的关注，用户界面越发受到用户的重视，清晰的逻辑及友好的交互已成每一款app的必备。As the killer feature of learning and education apps, the strength of the photo search function is an important criterion for judging whether a software is easy to use. The real charm of Internet + education lies in its ability to ignore objective factors such as geography, distance, and time, so that students can learn independently through apps. The interactive learning function of the online education platform is also a key point. Learners can find learners of the same level as themselves through the app to exchange learning and compete with each other. Similarly, no matter how powerful the function is, it is difficult to get the attention of students without a beautiful interface. The user interface is more and more valued by users, and clear logic and friendly interaction have become a must for every app.
为了实现上述功能，涉及到与终端底层的硬件交互，如摄像头、传感器、麦克风以及内部存储等。但是，当前移动终端设备有各种品牌，拥有不同的系统平台，目前大部分企业需要为在线教育移动终端开发适合多种平台的软件，需要根据不同平台的API进行原生应用的开发，也就是说，为了适配不同的操作系统，企业开发同一款应用就需要用到至少2到3个领域的程序员，而且每个平台的版本可能还会有一些差别，分别在各自的移动操作系统上进行原生应用的开发，会带来昂贵的开发费用、耗费大量人力、增加修改维护难度，若仅仅对部分功能进行细微修改，也不能避免在各个操作系统上单独进行版本改动。这就直接导致了用户体验下降，开发难度上升，以及开发进度不统一，反而提高了开发一款应用的成本。如果有种技术可以击穿这种壁垒，实现“一次编码，四处运行”，对于节约开发成本、提高开发效率都有很关键的作用。于此同时，可以降低开发的门槛，可以引入更多的开发者，提高初学者的开发热情。In order to realize the above functions, it involves interacting with the underlying hardware of the terminal, such as cameras, sensors, microphones, and internal storage. However, there are currently various brands of mobile terminal equipment and different system platforms. At present, most enterprises need to develop software suitable for multiple platforms for online education mobile terminals, and need to develop native applications according to APIs of different platforms. That is to say , in order to adapt to different operating systems, an enterprise needs to use programmers from at least 2 to 3 fields to develop the same application, and there may be some differences in the versions of each platform, which are carried out on their respective mobile operating systems. The development of native applications will bring expensive development costs, consume a lot of manpower, and increase the difficulty of modification and maintenance. If only minor modifications are made to some functions, version changes on each operating system cannot be avoided. This directly leads to a decline in user experience, an increase in development difficulty, and inconsistent development progress, which instead increases the cost of developing an application. If there is a technology that can break through this barrier and realize "code once, run everywhere", it will play a key role in saving development costs and improving development efficiency. At the same time, the development threshold can be lowered, more developers can be introduced, and the development enthusiasm of beginners can be increased.
为了降低开发成本，缩短开发周期，就需要寻求一种跨平台的移动应用开发解决方案。当前主流的跨移动平台应用开发解决方案之一是利用跨移动平台开发框架，如Apache Cordova、Titanium、Corona等，这类应用能够抽取出大部分公共模块，并结合对本地操作系统API的支持，比较节省开发成本。对于在线教育移动终端，单纯利用开发框架，也未能很好地满足移动终端应用程序的功能实现。为此，本发基于Cordova框架进行扩展，以满足移动教育终端设计更复杂的功能需求。In order to reduce the development cost and shorten the development cycle, it is necessary to seek a cross-platform mobile application development solution. One of the current mainstream cross-mobile platform application development solutions is to use cross-mobile platform development frameworks, such as Apache Cordova, Titanium, Corona, etc. This type of application can extract most of the public modules, combined with support for the local operating system API, Relatively save development costs. For online education mobile terminals, simply using the development framework cannot satisfy the function realization of mobile terminal applications. For this reason, the present invention is extended based on the Cordova framework to meet the more complex functional requirements of mobile education terminal design.
发明内容Contents of the invention
本发明的目的在于针对上述在线教育移动终端应用的多平台开发问题，提供一种基于Cordova的跨平台在线教育移动终端设计方法。The purpose of the present invention is to provide a Cordova-based cross-platform online education mobile terminal design method for the multi-platform development problem of the above-mentioned online education mobile terminal application.
为了实现上述目的，本发明采用的技术方案如下。In order to achieve the above object, the technical scheme adopted by the present invention is as follows.
一种基于Cordova的跨平台在线教育移动终端，包括：(1)Cordova框架模块；(2)Web层逻辑模块；(3)拍照搜题扩展模块；(4)视频录制与自识别扩展模块；(5)摇一摇功能扩展模块。A Cordova-based cross-platform online education mobile terminal, comprising: (1) a Cordova framework module; (2) a Web layer logic module; (3) an extension module for taking photos and searching questions; (4) an extension module for video recording and self-identification; 5) Shake the function expansion module.
整个终端通过Cordova框架支撑，分为Web层、Cordova层以及Native适配层，其中Native适配层提供底层功能支持，Cordova层包括各所述扩展模块；Web层提供各扩展模块的Javascript封装，以及上层的功能逻辑。The entire terminal is supported by the Cordova framework and is divided into a Web layer, a Cordova layer, and a Native adaptation layer, wherein the Native adaptation layer provides underlying function support, and the Cordova layer includes each of the extension modules; the Web layer provides the Javascript encapsulation of each extension module, and The functional logic of the upper layer.
进一步地，所述Cordova框架模块，包含Cordova核心组件以及插件，用于支持访问移动设备基本固件，提供Web层Javascript与Native层交互的桥梁，向上提供Javascript接口调用，并能在Native适配层执行完毕之后，通过Callback回调Javascript中的回调函数。Further, the Cordova framework module includes Cordova core components and plug-ins, which are used to support access to the basic firmware of mobile devices, provide a bridge for the interaction between Javascript on the Web layer and the Native layer, provide Javascript interface calls upward, and can be executed in the Native adaptation layer After completion, call back the callback function in Javascript through Callback.
进一步地，所述Web层逻辑模块，包含基于HTML、CSS、Javascript技术的用户界面模块、相应的页面逻辑模块、可以调用底层Native功能的Cordova调用模块，同时包含对前面所述扩展模块的Javascript接口封装。Further, the Web layer logic module includes a user interface module based on HTML, CSS, Javascript technology, a corresponding page logic module, a Cordova calling module that can call the underlying Native function, and includes a Javascript interface to the aforementioned extension module encapsulation.
进一步地，所述拍照搜题扩展模块，具体包含拍照模块、图片文字识别模块与内容搜索模块。Further, the extended module of taking pictures and searching questions specifically includes a taking pictures module, a picture and text recognition module and a content search module.
上述拍照模块，通过调用Cordova的camera插件模块功能完成对本地移动设备的摄像头调用，并传递获取的图片给图片文字识别模块。The camera module mentioned above completes the call to the camera of the local mobile device by calling the function of the camera plug-in module of Cordova, and transfers the obtained picture to the picture text recognition module.
上述图片文字识别模块，通过接入第三方ORC文字识别接口，对拍照获取的图片进行文字识别，获取识别文字结果，并传递给内容搜索模块。The above-mentioned image text recognition module, by accessing the third-party ORC text recognition interface, performs text recognition on the pictures obtained by taking pictures, obtains text recognition results, and transmits them to the content search module.
上述内容搜索模块，将文字识别的字符作为关键字，调用搜索引擎搜索整个题库，并根据相关度返回相应的题目信息。The above-mentioned content search module uses the characters recognized by the text as keywords, calls the search engine to search the entire question bank, and returns corresponding question information according to the degree of relevance.
进一步地，所述视频录制与自识别扩展模块，具体包含视频录制模块、语音文字识别模块、自动字幕模块与自动分类模块。Further, the video recording and self-recognition expansion module specifically includes a video recording module, a speech and text recognition module, an automatic subtitle module and an automatic classification module.
上述视频录制模块，通过Cordova的MediaCapture插件模块功能，调用视频捕捉函数来进行视频录制，并返回录制的结果。The above-mentioned video recording module uses the MediaCapture plug-in module function of Cordova to call the video capture function to perform video recording, and returns the recording result.
上述语音文字识别模块，通过封装第三方ASR语音识别SDK并调用相关接口，对录制的视频进行语音文字识别，并返回带时间戳的识别结果。The above speech and text recognition module, by encapsulating the third-party ASR speech recognition SDK and calling the relevant interface, performs speech and text recognition on the recorded video, and returns the recognition result with a time stamp.
上述自动字幕模块，对语音文字识别结果进行格式化处理，生成视频字幕文件，并提供预览与校正功能。The above-mentioned automatic subtitle module performs formatting processing on the speech and text recognition results, generates video subtitle files, and provides preview and correction functions.
上述自动分类模块，对语音文字识别结果进行关键字对比，自动分析获取视频的主要内容关键字，添加视频标签，对比知识目录对视频分类。The above-mentioned automatic classification module compares the keywords of the speech and text recognition results, automatically analyzes and obtains the main content keywords of the video, adds video tags, and compares the knowledge catalog to classify the videos.
进一步地，所述摇一摇功能扩展模块，具体包括：摇一摇模块、定位模块与匹配模块。Further, the shake-shake function expansion module specifically includes: a shake-shake module, a positioning module and a matching module.
上述摇一摇模块，通过Cordova的DeviceMotion插件模块功能，对移动设备的加速度传感器进行监听，当触发用户触发“摇一摇”动作时，返回一个标志。The above-mentioned shake-shake module monitors the acceleration sensor of the mobile device through the DeviceMotion plug-in module function of Cordova, and returns a sign when the user triggers the "shake-shake" action.
上述定位模块，通过Cordova的Geolocation插件模块功能，获取移动设备的GPS信息，并返回该结果。The above positioning module obtains the GPS information of the mobile device through the function of the Geolocation plug-in module of Cordova, and returns the result.
上述匹配模块，将所有“摇一摇”地用户的位置信息上传到云端，同时通过云端获取该类用户的个人学习情况，然后将处在一定距离内、短时间内也摇过的人进行配对。The above-mentioned matching module uploads the location information of all "shaking" users to the cloud, and at the same time obtains the personal learning situation of such users through the cloud, and then matches people who are within a certain distance and have shaken in a short time .
进一步地，本发明基于Cordova的跨平台在线教育移动终端的具体扩展模块设计如下：Further, the concrete expansion module design of the cross-platform online education mobile terminal based on Cordova of the present invention is as follows:
(一)拍照搜题扩展模块(1) Expansion module for camera search
此模块封装了Cordova的照相功能、图像文字识别功能、内容搜索匹配功能，完成拍照搜题的整个需求，并提供在Web层通过JavaScript调用该模块的接口。This module encapsulates Cordova's camera function, image text recognition function, content search and matching function, completes the entire demand for camera search, and provides an interface for calling this module through JavaScript on the Web layer.
该模块首先集成Cordova的camera插件，然后获取一个全局对象navigator.camera，通过该对象方法getPicture来获取图像，包括通过系统摄像头抓取图像和直接选取本地存储的图像。图像数据获取成功后，该数据被发送到onSuccess回调函数，在该函数内处理图像数据，并将其作为参数传入第三方ORC文字识别接口，按照第三方API接口调用说明调用，如通过向第三方API服务地址使用POST发送请求。最后接收请求的结果，获取图像中的文字识别结果，然后再将其作为参数，基于本地或第三方题库，通过搜索引擎进行相似度匹配，并根据相似度按序返回匹配题目，最后格式化显示与输出。This module first integrates the camera plug-in of Cordova, and then obtains a global object navigator.camera, and obtains images through the object method getPicture, including capturing images through the system camera and directly selecting locally stored images. After the image data is successfully obtained, the data is sent to the onSuccess callback function, the image data is processed in this function, and it is passed as a parameter to the third-party ORC text recognition interface, which is called according to the third-party API interface call instructions. The third-party API service address uses POST to send requests. Finally, receive the result of the request, obtain the text recognition result in the image, and then use it as a parameter, based on the local or third-party question bank, perform similarity matching through the search engine, and return the matching questions in order according to the similarity, and finally format and display with output.
基于本模块，上层的Web代码可以利用Javascript调用来实现拍照搜题功能，而Javascript的接口封装通过Cordova的exec()接口与底层通信，通过Plugin的实际Native代码段来实现具体平台上的功能,同时预留接入第三方ORC服务的抽象接口，方便接入第三方ORC服务。Based on this module, the upper-layer Web code can use Javascript calls to realize the function of taking photos and searching questions, and the interface packaging of Javascript communicates with the bottom layer through the exec() interface of Cordova, and realizes the functions on the specific platform through the actual Native code segment of Plugin. At the same time, an abstract interface for accessing third-party ORC services is reserved to facilitate access to third-party ORC services.
(二)视频录制与自识别扩展模块(2) Video recording and self-identification expansion module
此模块封装了Cordova的音视频录制功能、语音文字识别功能、自动生成视频字幕与分类信息功能，同时提供在Web层通过JavaScript调用该模块的接口。This module encapsulates Cordova's audio and video recording functions, speech and text recognition functions, automatic generation of video subtitles and classification information functions, and provides an interface for calling this module through JavaScript on the Web layer.
该模块首先集成Cordova的MediaCapture插件，获取到navigator.device.capture全局对象，调用Native层的录像与录音功能进行视频与音频录制。其中CaptureCallback函数作为方法参数传入，在该函数内完成录制视频与音频的处理，获取视频中的音频并发送到第三方ASR语音识别接口，通过标准JSON格式串行化语音数据和其他参数，并使用POST方式上传到第三方ASR服务API请求地址，最后通过服务器返回的callback函数获取回调地址，取回识别结果。将该识别结果分别与预定义的标签关键字与分类关键字进行对比匹配，根据重复度与相关度生成该视频的标签生成视频字幕信息与相应的知识章节信息，同时逐句格式化识别结果，添加时间戳与相应格式化信息，按播放时具体语言与样式需求生成视频字幕文件，作为外挂字幕在视频播放时进行载入。This module first integrates Cordova's MediaCapture plug-in, obtains the navigator.device.capture global object, and calls the recording and recording functions of the Native layer for video and audio recording. The CaptureCallback function is passed in as a method parameter. In this function, the recording video and audio processing is completed, the audio in the video is obtained and sent to the third-party ASR speech recognition interface, and the voice data and other parameters are serialized through the standard JSON format, and Use the POST method to upload to the third-party ASR service API request address, and finally obtain the callback address through the callback function returned by the server, and retrieve the recognition result. The recognition results are compared and matched with the predefined tag keywords and classification keywords, and the tags of the video are generated according to the degree of repetition and relevance to generate video subtitle information and corresponding knowledge chapter information, and the recognition results are formatted sentence by sentence. Add timestamps and corresponding formatting information, generate video subtitle files according to the specific language and style requirements during playback, and load them as external subtitles during video playback.
本模块的Javascript的接口封装通过Cordova的exec()接口与底层通信，通过Plugin的实际Native代码段来实现具体平台上的功能，同时预留接入第三方ASR服务的抽象接口，方便接入第三方ASR服务。在Web层利用Javascript调用来实现视频录制与自识别扩展功能。The Javascript interface encapsulation of this module communicates with the bottom layer through the exec() interface of Cordova, realizes the functions on the specific platform through the actual Native code segment of Plugin, and reserves the abstract interface for accessing third-party ASR services to facilitate access to the third-party Three-party ASR service. In the Web layer, Javascript calls are used to realize video recording and self-identification extension functions.
(三)摇一摇功能扩展模块(3) Shake function extension module
此模块封装了Cordova的加速度传感器监听功能与地理位置监听功能、摇一摇配对功能，同时提供在Web层通过JavaScript调用该模块的接口。This module encapsulates Cordova's acceleration sensor monitoring function, geographic location monitoring function, and shake-shake pairing function, and provides an interface for calling this module through JavaScript on the Web layer.
该模块集成Cordova的DeviceMotion插件与Geolocation插件，生成全局对象navigator.accelerometer和navigator.geolocation。首先通过navigator.accelerometer对象获取并监听设备当前物理加速度数据，并在监听事件中根据任一方向的加速度数据浮动幅度判定是否构成“摇一摇”的情形。达成“摇一摇”事件后，再通过navigator.geolocation对象获取用户当前地理位置，传入onSuccess回调函数，在该函数中，处理获取的地理位置信息并上传到云端。最后在云端按照地区与学习情况对一定事件内参与“摇一摇”的用户进行分类，同类之间随机相互配对，并将配对结果返回。This module integrates Cordova's DeviceMotion plugin and Geolocation plugin to generate global objects navigator.accelerometer and navigator.geolocation. First, obtain and monitor the current physical acceleration data of the device through the navigator.accelerometer object, and determine whether the situation of "shaking" is constituted according to the fluctuation range of the acceleration data in any direction during the monitoring event. After the "shake" event is achieved, the user's current geographic location is obtained through the navigator.geolocation object, and passed to the onSuccess callback function. In this function, the obtained geographic location information is processed and uploaded to the cloud. Finally, in the cloud, users who participated in "Shake" in a certain event are classified according to the region and learning situation, and the users of the same category are randomly matched with each other, and the matching results are returned.
基于本模块，上层的Web代码可以利用Javascript调用来实现摇一摇配对功能，通过Cordova的exec()接口实现Native代码段来实现具体平台上的功能，并封装为JavaScript调用接口供上层调用。Based on this module, the upper-layer Web code can use Javascript calls to realize the shake-shake pairing function, implement the Native code segment through Cordova's exec() interface to realize the functions on the specific platform, and encapsulate it as a JavaScript call interface for upper-layer calls.
本发明的有益效果：Beneficial effects of the present invention:
由上述技术方案可知，本发明提供的跨平台在线教育移动终端设计方法，基于开源的Cordova跨移动平台开发框架，具有以下效果：As can be seen from the above technical solution, the cross-platform online education mobile terminal design method provided by the present invention, based on the open-source Cordova cross-mobile platform development framework, has the following effects:
其一是使用HTML，CSS，JavaScript开发移动应用，降低移动前端开发的技术门槛；One is to use HTML, CSS, JavaScript to develop mobile applications, lowering the technical threshold of mobile front-end development;
其二是使用Cordova框架插件，使得即使用前端编程语言，也拥有和移动终端原生应用一样的访问移动设备基本固件的能力，如照相机、传感器等。The second is to use the Cordova framework plug-in, so that even if the front-end programming language is used, it has the same ability to access the basic firmware of the mobile device as the native application of the mobile terminal, such as cameras and sensors.
其三是跨平台性好，满足了一套开发程序可在多个不同的移动操作系统之上运行，降低对各个操作系统分别适配的难题，如果要增加对新的移动平台的支持，只需要对扩展模块进行平台适配工作，即可以共享上层逻辑模块，所以拥有很强的平台扩展性。The third is good cross-platform, which satisfies the need for a set of development programs to run on multiple different mobile operating systems, reducing the difficulty of adapting to each operating system separately. If you want to increase support for new mobile platforms, you only need to It is necessary to adapt the platform to the expansion module, that is, the upper logic module can be shared, so it has strong platform scalability.
其四是使用Cordova封装的插件与终端底层硬件交互，大大简化了移动在线教育应用与本地操作系统API交互的编码工作，可以直接通过JavaScript接口调用扩展模块完成相应的功能，降低开发成本，缩短开发周期。The fourth is to use the Cordova-encapsulated plug-in to interact with the underlying hardware of the terminal, which greatly simplifies the coding work of the interaction between the mobile online education application and the local operating system API, and can directly call the extension module through the JavaScript interface to complete the corresponding functions, reducing development costs and shortening the development time. cycle.
附图说明Description of drawings
为了更清楚地说明本发明实施实例，下面将对实施例或现有技术描述中所需要使用的附图作一简单地介绍。In order to illustrate the implementation examples of the present invention more clearly, a brief introduction will be given below to the accompanying drawings that are required in the description of the embodiments or the prior art.
图1为跨平台在线教育移动终端的主要框架模块示意图。Figure 1 is a schematic diagram of main framework modules of a cross-platform online education mobile terminal.
图2为跨平台在线教育移动终端的总体系架构示意图。Fig. 2 is a schematic diagram of the overall architecture of a cross-platform online education mobile terminal.
图3为拍照搜题扩展模块的架构示意图。Figure 3 is a schematic diagram of the structure of the camera search extension module.
图4为照视频录制与自识别扩展模块的架构示意图。Figure 4 is a schematic diagram of the architecture of the video recording and self-identification expansion module.
图5为摇一摇功能扩展模块的架构示意图。Fig. 5 is a schematic diagram of the architecture of the shake-shake function extension module.
具体实施方式detailed description
为使本发明实施例的目的、技术方案和优点更加清楚，下面将结合本发明提供的附图，对本发明实施例中的技术方案进行清楚、完整地描述，基于本发明中的实施实例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施实例，都属于本发明保护的范围。In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings provided by the present invention. Based on the implementation examples in the present invention, this All other implementation examples obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.
本实例提供一种基于Cordova的跨平台在线教育移动终端，框架模块如图1所示，包括：(1)Cordova框架模块；(2)Web层逻辑模块；(3)拍照搜题扩展模块；(4)视频录制与自识别扩展模块；(5)摇一摇功能扩展模块。This example provides a cross-platform online education mobile terminal based on Cordova. The framework modules are shown in Figure 1, including: (1) Cordova framework module; (2) Web layer logic module; (3) photo search question extension module; ( 4) Video recording and self-identification extension module; (5) Shake function extension module.
终端体系架构如图2所示，整个终端通过Cordova框架支撑，分为Web层、Cordova层以及Native适配层，其中Native适配层提供底层功能支持，Cordova层包括各所述扩展模块；Web层提供各扩展模块的Javascript封装，以及上层的功能逻辑。The terminal system architecture is shown in Figure 2. The entire terminal is supported by the Cordova framework and is divided into a Web layer, a Cordova layer, and a Native adaptation layer. The Native adaptation layer provides underlying functional support, and the Cordova layer includes the above-mentioned extension modules; the Web layer Provides the Javascript encapsulation of each extension module and the functional logic of the upper layer.
具体地，Cordova框架模块，包含Cordova核心组件以及插件，用于支持访问移动设备基本固件，提供Web层Javascript与Native层交互的桥梁，向上提供Javascript接口调用，并能在Native适配层执行完毕之后，通过Callback回调Javascript中的回调函数。Specifically, the Cordova framework module, including Cordova core components and plug-ins, is used to support access to the basic firmware of mobile devices, provide a bridge for the interaction between Javascript on the Web layer and the Native layer, and provide Javascript interface calls upwards, and can be executed after the Native adaptation layer is completed. , call back the callback function in Javascript through Callback.
具体地，所述Web层逻辑模块，包含基于HTML、CSS、Javascript技术的用户界面模块、相应的页面逻辑模块、可以调用底层Native功能的Cordova调用模块，同时包含对前面所述扩展模块的Javascript接口封装。Specifically, the Web layer logic module includes a user interface module based on HTML, CSS, Javascript technology, a corresponding page logic module, a Cordova calling module that can call the underlying Native function, and includes a Javascript interface to the aforementioned extension module encapsulation.
进一步地，本发明基于Cordova的跨平台在线教育移动终端的扩展模块具体设计步骤如下：Further, the specific design steps of the expansion module of the cross-platform online education mobile terminal based on Cordova in the present invention are as follows:
(一)拍照搜题扩展模块(1) Expansion module for camera search
此模块具体包含拍照模块、图片文字识别模块与内容搜索模块，封装了Cordova的照相功能、图像文字识别功能、内容搜索匹配功能，完成拍照搜题的整个需求，并提供在Web层通过JavaScript调用该模块的接口，Javascript的接口封装通过Cordova的exec()接口与底层通信，在exec接口内通过native代码完成各个具体平台相应的功能。根据图3拍照搜题扩展模块的架构图，具体模块设计如下：This module specifically includes a camera module, a picture text recognition module and a content search module, which encapsulates Cordova's camera function, image text recognition function, and content search and matching functions, and completes the entire demand for photo search and search questions, and provides JavaScript in the Web layer. The interface of the module, the interface encapsulation of Javascript communicates with the bottom layer through the exec() interface of Cordova, and completes the corresponding functions of each specific platform through the native code in the exec interface. According to the architecture diagram of the extended module of photo search and question search in Figure 3, the specific module design is as follows:
拍照模块：首先集成cordova-plugin-camera插件，然后获取一个全局对象navigator.camera，通过该对象方法getPicture来获取图像，包括通过系统摄像头抓取图像和直接选取本地存储的图像，模块返回该图像并通过onSuccess回调函数传递给下一模块。Camera module: first integrate the cordova-plugin-camera plug-in, then obtain a global object navigator.camera, and obtain images through the object method getPicture, including capturing images through the system camera and directly selecting locally stored images, the module returns the image and Pass it to the next module through the onSuccess callback function.
图片文字识别模块：图像数据获取成功后，该数据被发送到onSuccess回调函数，在该函数内处理图像数据，并创建一个第三方API调用的抽象接口，将图像数据作为参数传入该第三方ORC文字识别接口，具体使用时再按照第三方API接口调用说明调用，如通过向第三方API服务地址使用POST发送请求。将第三方服务接口调用的处理结果作为模块输出，再次传递给下一模块。Image text recognition module: After the image data is successfully obtained, the data is sent to the onSuccess callback function, the image data is processed in this function, and an abstract interface for third-party API calls is created, and the image data is passed as a parameter to the third-party ORC For the text recognition interface, call it according to the third-party API interface call instructions when using it, such as sending a request to the third-party API service address using POST. The processing result of the third-party service interface call is output as a module and passed to the next module again.
内容搜索模块：从第三方服务接口调用的返回值来接收请求的结果，获取图像中的文字识别结果，然后再将其作为参数，基于本地或第三方题库，通过搜索引擎进行相似度匹配，并根据相似度按序返回匹配题目，最后格式化显示与输出。Content search module: Receive the result of the request from the return value of the third-party service interface call, obtain the text recognition result in the image, and then use it as a parameter to perform similarity matching through the search engine based on the local or third-party question bank, and Matching topics are returned in order according to the similarity, and finally formatted for display and output.
(二)视频录制与自识别扩展模块(2) Video recording and self-identification expansion module
此模块具体包含视频录制模块、语音文字识别模块、自动字幕模块与自动分类模块，封装了Cordova的音视频录制功能、语音文字识别功能、自动生成视频字幕与分类信息功能，也提供在Web层通过JavaScript调用该模块的接口，其Javascript的接口封装同上一个模块。根据图4照视频录制与自识别扩展模块的架构图，具体模块设计如下：This module specifically includes a video recording module, a speech and text recognition module, an automatic subtitle module, and an automatic classification module. JavaScript calls the interface of this module, and its Javascript interface encapsulation is the same as that of the previous module. According to the architecture diagram of the video recording and self-identification expansion module in Figure 4, the specific module design is as follows:
视频录制模块:首先集成cordova-plugin-media-capture插件，获取到navigator.device.capture全局对象，使用对象方法captureVideo与captureAudio别调用Native层的录像与录音功能进行视频与音频录制。CaptureCallback回调函数作为方法参数，接收并处理录制的音视频。Video recording module: first integrate the cordova-plugin-media-capture plug-in, obtain the navigator.device.capture global object, and use the object methods captureVideo and captureAudio to call the recording and recording functions of the Native layer for video and audio recording. The CaptureCallback callback function is used as a method parameter to receive and process the recorded audio and video.
语音文字识别模块：在CaptureCallback函数内部完成录制视频与音频的处理，获取视频中的音频，然后引入第三方ASR语音识别接口，通过标准JSON格式串行化语音数据和其他参数，并使用POST方式上传到第三方ASR服务API请求地址，最后通过服务器返回的callback函数获取回调地址，取回识别结果。Speech and text recognition module: complete the recording video and audio processing within the CaptureCallback function, obtain the audio in the video, and then introduce the third-party ASR speech recognition interface, serialize the voice data and other parameters in the standard JSON format, and upload them using POST Go to the third-party ASR service API request address, and finally obtain the callback address through the callback function returned by the server, and retrieve the recognition result.
自动分类模块：预定义多个知识点关键字，然后将第三方ASR服务处理识别结果分别与预定义的关键字进行对比匹配，根据重复度与相关度，按照一定的关键字组合规则，生成该视频的标签生成视频标签信息，并归类到相应的知识章节目录。Automatic classification module: pre-define multiple knowledge point keywords, and then compare and match the identification results of third-party ASR service processing with the predefined keywords, and generate the The video tag generates video tag information and classifies it into the corresponding knowledge chapter directory.
自动字幕模块：逐句格式化第三方ASR服务处理识别结果，添加时间戳与相应格式化信息，按播放时具体语言与样式需求生成视频字幕文件，作为外挂字幕在视频播放时进行载入。Automatic subtitle module: Format third-party ASR service processing recognition results sentence by sentence, add timestamp and corresponding formatting information, generate video subtitle files according to specific language and style requirements during playback, and load them as external subtitles during video playback.
(三)摇一摇功能扩展模块(3) Shake function extension module
此模块具体包括：摇一摇模块、定位模块与匹配模块，封装了Cordova的加速度传感器监听功能与地理位置监听功能、摇一摇配对功能，同时提供在Web层通过JavaScript调用该模块的接口。根据图5摇一摇功能扩展模块的架构图，具体模块设计如下：This module specifically includes: shake-shake module, positioning module and matching module, which encapsulates the acceleration sensor monitoring function and geographic location monitoring function of Cordova, shake-shake pairing function, and provides an interface for calling this module through JavaScript on the Web layer. According to the architecture diagram of the shake function expansion module in Figure 5, the specific module design is as follows:
摇一摇模块：集成cordova-plugin-device-motion插件，生成navigator.accelerometer全局对象。然后，通过navigator.accelerometer对象的getCurrentAcceleration方法与watchAcceleration方法获取并监听设备当前物理加速度数据，并在监听事件中根据任一方向的加速度数据浮动幅度判定是否构成“摇一摇”的情形，对象方法接收一个callback回调函数作为参数，用以执行监听到“摇一摇”事件时的后续逻辑。Shake Module: Integrate the cordova-plugin-device-motion plug-in to generate a navigator.accelerometer global object. Then, use the getCurrentAcceleration method and watchAcceleration method of the navigator.accelerometer object to obtain and monitor the current physical acceleration data of the device, and determine whether it constitutes a "shake" situation according to the acceleration data floating range in any direction during the monitoring event. The object method receives A callback callback function is used as a parameter to execute the follow-up logic when the "shake" event is heard.
定位模块：集成cordova-plugin-geolocation插件，生成navigator.geolocation全局对象，达成“摇一摇”事件后，通过navigator.geolocation对象的getCurrentPosition方法获取用户当前地理位置，传入onSuccess回调函数，在该函数中，处理获取的地理位置信息并上传到云端。Positioning module: Integrate the cordova-plugin-geolocation plug-in to generate the navigator.geolocation global object. After the "shake" event is reached, the user's current geographic location is obtained through the getCurrentPosition method of the navigator.geolocation object, and the onSuccess callback function is passed in. , process the acquired geographic location information and upload it to the cloud.
匹配模块：在云端按照地区与学习情况对一定事件内参与“摇一摇”的用户进行分类，同类之间随机相互配对，并将配对结果返回。Matching module: Classify the users who participated in the "Shake" in a certain event according to the region and learning situation in the cloud, randomly match each other with each other in the same category, and return the matching results.
以上所述仅为本发明的优先实施方式，同时也只是本发明的其中一个具体实例，本发明并不限定于上述实施方式，只要以基本相同手段实现本发明目的的技术方案都属于本发明的保护范围之内。The above is only a preferred embodiment of the present invention, and it is also only one of the specific examples of the present invention. The present invention is not limited to the above-mentioned embodiment, as long as the technical solutions that achieve the purpose of the present invention by basically the same means belong to the present invention within the scope of protection.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710896277.0ACN107479906A (en) | 2017-09-28 | 2017-09-28 | cross-platform online education mobile terminal based on Cordova |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710896277.0ACN107479906A (en) | 2017-09-28 | 2017-09-28 | cross-platform online education mobile terminal based on Cordova |
| Publication Number | Publication Date |
|---|---|
| CN107479906Atrue CN107479906A (en) | 2017-12-15 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710896277.0APendingCN107479906A (en) | 2017-09-28 | 2017-09-28 | cross-platform online education mobile terminal based on Cordova |
| Country | Link |
|---|---|
| CN (1) | CN107479906A (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110365738A (en)* | 2019-05-31 | 2019-10-22 | 平安国际智慧城市科技股份有限公司 | Localization method, device and storage medium based on Hybird framework |
| EP3817395A1 (en)* | 2019-10-30 | 2021-05-05 | Beijing Xiaomi Mobile Software Co., Ltd. | Video recording method and apparatus, device, and readable storage medium |
| CN113608797A (en)* | 2021-08-04 | 2021-11-05 | 南方电网数字电网研究院有限公司 | Native plug-in extension system based on nodejs and cordova |
| CN114143032A (en)* | 2021-11-01 | 2022-03-04 | 北京银盾泰安网络科技有限公司 | SSH-based PC (personal computer) end mutual access system and interaction method thereof |
| CN116166224A (en)* | 2022-12-29 | 2023-05-26 | 泰豪软件股份有限公司 | A method and system for developing hybrid App based on Cordova |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090313026A1 (en)* | 1998-10-02 | 2009-12-17 | Daniel Coffman | Conversational computing via conversational virtual machine |
| CN102902472A (en)* | 2012-05-07 | 2013-01-30 | 腾讯科技（深圳）有限公司 | Method and system for triggering commands and method and system for recommending users |
| CN105426518A (en)* | 2015-12-03 | 2016-03-23 | 广东小天才科技有限公司 | Method and system for searching questions by photographing |
| CN105573129A (en)* | 2015-11-10 | 2016-05-11 | 华南理工大学 | Cordova-based cross-platform smart home mobile terminal |
| US20160253171A1 (en)* | 2015-02-26 | 2016-09-01 | Yan Zang | Application development system with native feature engine |
| CN107124647A (en)* | 2017-05-27 | 2017-09-01 | 深圳市酷开网络科技有限公司 | A kind of panoramic video automatically generates the method and device of subtitle file when recording |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090313026A1 (en)* | 1998-10-02 | 2009-12-17 | Daniel Coffman | Conversational computing via conversational virtual machine |
| CN102902472A (en)* | 2012-05-07 | 2013-01-30 | 腾讯科技（深圳）有限公司 | Method and system for triggering commands and method and system for recommending users |
| US20160253171A1 (en)* | 2015-02-26 | 2016-09-01 | Yan Zang | Application development system with native feature engine |
| CN105573129A (en)* | 2015-11-10 | 2016-05-11 | 华南理工大学 | Cordova-based cross-platform smart home mobile terminal |
| CN105426518A (en)* | 2015-12-03 | 2016-03-23 | 广东小天才科技有限公司 | Method and system for searching questions by photographing |
| CN107124647A (en)* | 2017-05-27 | 2017-09-01 | 深圳市酷开网络科技有限公司 | A kind of panoramic video automatically generates the method and device of subtitle file when recording |
| Title |
|---|
| 肖晨: "移动学习APP的实现与应用", 《科技广场》* |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110365738A (en)* | 2019-05-31 | 2019-10-22 | 平安国际智慧城市科技股份有限公司 | Localization method, device and storage medium based on Hybird framework |
| EP3817395A1 (en)* | 2019-10-30 | 2021-05-05 | Beijing Xiaomi Mobile Software Co., Ltd. | Video recording method and apparatus, device, and readable storage medium |
| CN113608797A (en)* | 2021-08-04 | 2021-11-05 | 南方电网数字电网研究院有限公司 | Native plug-in extension system based on nodejs and cordova |
| CN114143032A (en)* | 2021-11-01 | 2022-03-04 | 北京银盾泰安网络科技有限公司 | SSH-based PC (personal computer) end mutual access system and interaction method thereof |
| CN114143032B (en)* | 2021-11-01 | 2023-07-07 | 北京银盾泰安网络科技有限公司 | PC end mutual access system based on SSH and interaction method thereof |
| CN116166224A (en)* | 2022-12-29 | 2023-05-26 | 泰豪软件股份有限公司 | A method and system for developing hybrid App based on Cordova |
| Publication | Publication Date | Title |
|---|---|---|
| CN107479906A (en) | cross-platform online education mobile terminal based on Cordova | |
| US11973993B2 (en) | Machine learning based media content annotation | |
| US10165307B2 (en) | Automatic recognition of entities in media-captured events | |
| CN113518256B (en) | Video processing method, video processing device, electronic equipment and computer readable storage medium | |
| WO2021068744A1 (en) | Insect identification method and system | |
| WO2014198132A1 (en) | Methods and systems for information matching | |
| US10929477B2 (en) | Environment information storage and playback method, storage and playback system and terminal | |
| CN112416984B (en) | A data processing method and device thereof | |
| CN114404960B (en) | Cloud game resource data processing method, device, computer equipment and storage medium | |
| WO2024082914A1 (en) | Video question answering method and electronic device | |
| WO2020044094A1 (en) | Resource recommendation method and apparatus, electronic device and computer readable medium | |
| WO2021115346A1 (en) | Media file processing method, device, readable medium, and electronic apparatus | |
| CN113301382A (en) | Video processing method, device, medium, and program product | |
| CN115103206A (en) | Video data processing method, apparatus, equipment, system and storage medium | |
| CN117557801A (en) | A method, device, storage medium and electronic equipment for surveying and mapping network assets | |
| CN104581639B (en) | A kind of seamless space-time cut-in method applied to mobile terminal sensor senses information | |
| CN112307823B (en) | Method and device for labeling objects in video | |
| CN114339074B (en) | Method, device, computer equipment and storage medium for generating comment video | |
| CN113407771B (en) | Monitoring and scheduling method, system, device and storage medium | |
| CN115461709A (en) | Hierarchical context-specific actions from ambient speech | |
| CN109255023A (en) | Hint information processing method and processing device | |
| CN108536343B (en) | Control display method and device, terminal and storage medium | |
| CN116052053A (en) | A method and device for improving the accuracy of surveillance images in a smart museum | |
| CN109040407A (en) | Voice acquisition method and device based on mobile terminal | |
| CN115774806A (en) | A search processing method, device, equipment, medium and program product |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication | Application publication date:20171215 |