Intelligent Cockpit Human-Machine Interaction Speech Dataset – New Energy Vehicles

1. Product Overview

The intelligent cockpit human–machine interaction speech dataset comprises a total of 10,854 hours. The recording language is Chinese, and the recordings were made in China. The transcribed text covers five categories, including mainstream in-vehicle smart control and command words. The text was designed to include both normal and fast speaking speeds. A total of 2,076 speakers from various accent regions across China participated in the recording process. Recordings were conducted in real in-vehicle environments, simulating autonomous driving scenarios, using seven high-fidelity microphones (16 kHz, 16-bit, 1207H × 7), an Android smartphone (16 kHz, 16-bit, 1199H), and an Apple smartphone (16 kHz, 16-bit, 1202H). This database has been professionally transcribed and annotated by skilled linguists and underwent rigorous quality assurance, achieving a text accuracy rate exceeding 95%.

2. Product Catalog Structure

Catalog Structure

Data directory structure

└─DOC (Text Description Files)
││├─wav_list.txt (Audio File List)
││├─content.txt (Transcription Content List)
││├─spk_info.xlsx (Speaker Information)
│
└─SPEECHDATA (Data Folder)
│├─V0001 (Speaker Folder)
│├─P1 (Recording Channel Number)
││ ││ V0001P10001.wav (Audio File)

Naming Convention

<CORPUS>/<USAGE>/<SPEAKER_NUM>/<POINT_ID>/<SENTENCE_ID> e.g., /SPEECHDATA/V0001/P1/V0001P10001.wav

Directory · Content · Notes

CORPUS · · Database Name and Identifier

USAGE · SPEECHDATA · Folder Name

SPEAKER_NUM · V0001 · Speaker Folder

POINT_ID · P1 · Recording Position ID

SPEECH_ID · V0001P10001.wav · WAV File

3. Text Design

Corpus Construction

Considering the applications of speech recognition in intelligent in-vehicle systems and autonomous driving, the corpus was selected across five domains. Each domain’s content was processed according to established rules.

Serial Number · Domain

1 · Voice Control

2 · Communication Command Words

3 · Multimedia

4 · POI

5 · Balanced Sentences

• De-identification: Removed politically sensitive, personally private, or sexually explicit/violent content.
• Removed symbols such as <, >, [ , ], ～, /, \, =.
• Excluded content containing languages other than Chinese and English.
• Eliminated sentences longer than 25 characters.
• Standardized formatting.

Text Structure

To ensure comprehensive voice coverage and phonetic balance, each audio file in this database contains approximately 508 sentences, drawn from the corpus pool. The distribution is as follows:

Category · Assigned Quantity (Number of Sentences) · Sentence Numbers

Voice Control · 35 · 0021~0055

Voice Control (Fast Speaking) · 35 · 0056~0090

Communication Command Words · 30 · 0091~0120

Communication Command Words (Fast Speaking) · 30 · 0121~0150

Multimedia · 35 · 0151~0185

Multimedia (Fast Speaking) · 35 · 0186~0220

Communication Command Words · 25 · 0221~0245

POI · 36 · 0246~0281

POI (Fast Speaking) · 36 · 0282~0317

Balanced Sentences · 200 · 0318~0517

Total · 497 ·

4. Recording Environment

Recording Site

The recording environment was carefully designed to simulate three vehicle speeds (low, medium, high), three window settings (closed, one-third open on the passenger side, one-third open on the driver’s side), two music states (on, off), and two air-conditioning conditions (off, on), resulting in a total of 28 distinct scenarios. High-speed scenarios did not include open-window conditions. Low speed: vehicle speed < 40 km/h; Medium speed: 40 km/h ≤ vehicle speed < 80 km/h; High speed: 80 km/h ≤ vehicle speed < 120 km/h.

Scenario 1 · Low Speed – Window Closed – Music Off – Air Conditioning Off · Scenario 13 · Medium Speed – Window Closed – Music Off – Air Conditioning Off · Scenario 25 · High Speed – Music Off – Air Conditioning Off

Scenario 2 · Low Speed – Window Closed – Music Off – Air Conditioning On · Scenario 14 · Medium Speed – Window Closed – Music Off – Air Conditioning On · Scenario 26 · High Speed – Music Off – Air Conditioning On

Scenario 3 · Low Speed – Window Closed – Music On – Air Conditioning Off · Scenario 15 · Medium Speed – Window Closed – Music On – Air Conditioning Off · Scenario 27 · High Speed – Music On – Air Conditioning Off

Scenario 4 · Low Speed – Window Closed – Music On – Air Conditioning On · Scenario 16 · Medium Speed – Window Closed – Music On – Air Conditioning On · Scenario 28 · High Speed – Music On – Air Conditioning On

Scenario 5 · Low Speed – Passenger Side Open – Music Off – Air Conditioning Off · Scenario 17 · Medium Speed – Passenger Side Open – Music Off – Air Conditioning Off · Scenario 28 · High Speed – Music Off – Air Conditioning On

Scenario 6 · Low Speed – Passenger Side Open – Music Off – Air Conditioning On · Scenario 18 · Medium Speed – Passenger Side Open – Music Off – Air Conditioning On · Scenario 28 · High Speed – Music Off – Air Conditioning On

Scenario 7 · Low Speed – Passenger Side Open – Music On – Air Conditioning Off · Scenario 19 · Medium Speed – Passenger Side Open – Music On – Air Conditioning Off · Scenario 28 · High Speed – Music On – Air Conditioning Off

Scenario 8 · Low Speed – Passenger Side Open – Music On – Air Conditioning On · Scenario 20 · Medium Speed – Passenger Side Open – Music On – Air Conditioning On

Scenario 9 · Low Speed – Driver Side Open – Music Off – Air Conditioning Off · Scenario 21 · Medium Speed – Driver Side Open – Music Off – Air Conditioning Off

Scenario 10 · Low Speed – Driver Side Open – Music Off – Air Conditioning On · Scenario 22 · Medium Speed – Driver Side Open – Music Off – Air Conditioning On

Scenario 11 · Low Speed – Driver Side Open – Music On – Air Conditioning Off · Scenario 23 · Medium Speed – Driver Side Open – Music On – Air Conditioning Off

Scenario 12 · Low Speed – Driver Side Open – Music On – Air Conditioning On · Scenario 24 · Medium Speed – Driver Side Open – Music On – Air Conditioning On

Recording Equipment

The recording equipment included high-fidelity microphones and recorders, along with Android and Apple smartphones. The high-fidelity recorder stored audio at 16 kHz, 16-bit mono, while the Android and Apple phones recorded at 16 kHz, 16-bit mono.

Recording Method

Within the vehicle, nine recording positions were set up: three close-range positions and six functional positions. Functional positions were fixed in front of the driver, near the rearview mirror, and on the center console, among others. Close-range positions were located 25 cm away from the speaker, equipped with a high-fidelity microphone, an Apple phone, and an Android phone. Speakers read aloud the provided transcripts at their normal speaking volume, following prompts for both normal and fast speaking speeds.

Position Number · Location · Description

Position 1 · Rearview Mirror · High-Fidelity Microphone

Position 2 · Rearview Mirror · High-Fidelity Microphone

Position 3 · Passenger-Side Handlebar · High-Fidelity Microphone

Position 4 · Passenger-Side Handlebar · High-Fidelity Microphone

Position 5 · Center Console Screen · High-Fidelity Microphone

Position 6 · Passenger-Side Front · High-Fidelity Microphone

Position 7 · Head-Mounted Device · High-Fidelity Microphone

Position 8 · 25 cm from Speaker, Right Passenger-Side Front · Apple Phone

Position 9 · 25 cm from Speaker, Passenger-Side Front · Android Phone

5. Annotation and Transcription Guidelines

Transcribers are tasked with accurately rendering the spoken content based on the audio they hear, striving to maintain complete consistency between the written text and the original pronunciation. Key guidelines include: 1) Transcriptions must precisely match the spoken content—no extra words, missing words, or errors. 2) Numerals should be written in Chinese characters, such as “一二三,” rather than “123.” Pay special attention to distinguishing “一” from “幺” and “二” from “两.” 3) English words appearing within sentences should be transcribed phonetically, such as “thank you.” Letters pronounced individually should be rendered in uppercase followed by a space—for example, “N B A” or “U F O.” 4) Symbols appearing in sentences should be transcribed according to the actual pronunciation of the speaker, such as “三W点百度点com.” Unpronounced symbols should be omitted. Brand names and proper nouns should be transcribed in their commonly used forms, such as “QQ空间,” “iPhone,” or “喜马拉雅.” 5) The completeness of annotations must align with the actual pronunciation; no omissions are permitted. 6) Fast‑spoken segments should be marked with “@@@” before and after the text—for example, “@@@ 烦请把温度调为十九度 @@@.” Text reflecting the recording environment should be enclosed by “######” markers—for instance, “###### 音乐开空调开######,” indicating that the text reflects the environmental conditions under which it was recorded.

6. Speaker Information

Basic Information Records

Speaker information includes task number, gender, accent region, age range, and place of origin.

Task Number · Gender · Accent Region · Age Range · Environmental Scenario

V0001 · Male · Northern Region · A · Low Speed – Window Closed – Music Off – Air Conditioning Off

Each speaker receives one unique task number, corresponding to a specific transcription assignment. Each speaker may participate in only one recording session. Gender: male—“男”; female—“女.” Accent region: categorized into Northern, Southern, and Other based on the speaker’s native dialect area. Age range: A (25–35 years old), B (36–45 years old). Environmental scenario: records the specific context of the recording. See the list of scenarios below.

Speaker Demographic Characteristics

Gender Ratio

The database contains a total of 2,076 speakers, including 1,470 males and 606 females.

Gender · Male · Female · Total

Ratio · 70.7% · 29.3% · 100%

Age Distribution

Age group A (25–35): 1,510 individuals; Age group B (36–45): 566 individuals.

Age Group · Age Bracket · Number of Individuals · Proportion

A · 25–35 Years Old · 1,510 · 73%

B · 36–45 Years Old · 566 · 27%

Total · 2,076 · 100%

Dialect Region Proportions

Northern region: 1,460 individuals; Southern region: 607 individuals; Other: 9 individuals.

Region · Number of Individuals · Proportion

Northern Region · 1,460 · 70.3%

Southern Region · 607 · 29.3%

Other Regions · 9 · 0.4%

Total · 2,076 · 100%

Sample Data Download

Click to download sample data