Intelligent Cockpit Human–Machine Interaction Speech Dataset – Internal Combustion Engine Vehicles
About this dataset
1. Product Overview
The intelligent cockpit human–machine interaction speech dataset comprises a total of 613 hours. The recording language is Chinese, and the recordings were made in China. The transcribed text covers ten categories, including mainstream in-vehicle smart controls and commands. Primarily featuring speakers from northern China, 300 speakers participated in the recording process. Recordings were conducted in real in-vehicle environments, simulating autonomous driving scenarios, using four high-fidelity microphones (44.1 kHz, 16-bit, 122 hours) and one smartphone (16 kHz, 16-bit, 124 hours). This database has been professionally transcribed and annotated by skilled linguists and underwent rigorous quality assurance, achieving a text accuracy rate exceeding 95%.
2. Product Catalog Structure
Catalog Structure
Data catalog structure
└─DOC (Text Description File)
││├─wav_list.txt (Audio List)
││├─content.txt (Transcription List)
││├─spk_info.xlsx (Speaker Information)
│
└─SPEECHDATA (Data Folder)
│├─0001 (Speaker Folder)
│├─CC0001A (Speaker Folder)
││ ││ CC0001A0001.wav (Audio File)
││ ││ CC0001A0001.txt (Speech Transcript)
Naming Convention
<CORPUS_ID><SPEAKER_IC><WAV_NUM> e.g., CC0001A0001.wav
Directory Name · Content · Notes
CORPUS_ID · CC (Chinese in Car) · Database Name
SPEAKER_NUM · 0001 · Speaker ID
POINT_ID · A\B\C\D\F · Recording Position ID
WAV_NUM · 0001 · WAV Number
3. Text Design
Corpus Construction
Considering the applications of speech recognition in intelligent in-vehicle systems and autonomous driving, the corpus was curated across ten distinct domains. Each domain was processed according to established guidelines.
Serial Number · Domain
1 · Wake-Up Commands
2 · Highway Information
3 · Administrative Geographical Information
4 · Commands and Control Statements
5 · Numeric Strings
6 · Music
7 · Radio
8 · Air Conditioning
9 · Map Coordinates
10 · General Content Text
- De-identification: Removal of politically sensitive, personally identifiable, or sexually explicit content.
- Symbol removal: Elimination of symbols such as <, >, [ , ] , ～ , / , \ , =.
- Language filtering: Exclusion of materials containing languages other than Chinese and English.
- Length restriction: Omission of sentences longer than 25 words.
- Standardization: Uniform formatting of all entries.
Text Structure
To ensure comprehensive phonetic coverage and balanced phoneme distribution, each audio file in this database contains approximately 508 sentences, drawn from the corpus pool. The structure is as follows:
Serial Number · Domain · Sentence Allocation per Domain
1 · General · 50 sentences
2 · Wake-Up Commands · 48 sentences
3 · Highway Information · 20 sentences
4 · Administrative Geographical Information · 30 sentences
5 · Commands and Control Statements · 35 sentences
6 · Numeric Strings · 40 sentences
7 · Music · 30 sentences
8 · Radio · 30 sentences
9 · Air Conditioning · 35 sentences
10 · Map Coordinates · 40 sentences
11 · General Content Text · 50 sentences
Total · 10 Domains · 508 Sentences
4. Recording Environment
Recording Site
In the in-vehicle setting, seven distinct scenarios were designed and recorded sequentially. The vehicle fleet included three compact cars, two midsize cars, and one large car.
Scenario Number · Vehicle Speed
A · Engine Off → Urban Slow Speed 0–40 km/h
B · Urban Medium Speed 40–80 km/h with one-third of the passenger-side window open
C · Highway 80–120 km/h with windows closed
D · Engine Off with one-third of the passenger-side window open → Urban Slow Speed 0–40 km/h
E · Engine Off with one-third of the driver-side window open → Urban Slow Speed 0–40 km/h
F · Urban Medium Speed 40–80 km/h with one-third of the passenger-side window open
G · Urban Medium Speed 40–80 km/h with one-third of the driver-side window open
Within each scenario, additional auxiliary conditions—such as music playback or air conditioning control—were integrated into the overall environment.
Scenario Number · Vehicle Speed · Auxiliary Conditions
A · Engine Off → Urban Slow Speed 0–40 km/h · Music/Air Conditioning
B · Urban Medium Speed 40–80 km/h with one-third of the passenger-side window open · Music/Air Conditioning
C · Highway 80–120 km/h with windows closed · Music/Air Conditioning
D · Engine Off with one-third of the passenger-side window open → Urban Slow Speed 0–40 km/h · Music/Air Conditioning
E · Engine Off with one-third of the driver-side window open → Urban Slow Speed 0–40 km/h · Music/Air Conditioning
F · Urban Medium Speed 40–80 km/h with one-third of the passenger-side window open · Music/Air Conditioning
G · Urban Medium Speed 40–80 km/h with one-third of the driver-side window open · Music/Air Conditioning
Music and air-conditioning states were further categorized into six groups and assigned to specific segments within the corpus.
Auxiliary Condition ·
Air Conditioning State · Music State
Air Conditioning Off · Music Off
Air Conditioning Off · Music On
Low Wind Air Conditioning On · Music Off
Low Wind Air Conditioning On · Music On
High Wind Air Conditioning On · Music Off
High Wind Air Conditioning On · Music On
Recording Equipment
The recording setup includes high-fidelity microphones, tape recorders, and smartphones. Data storage formats comprise high-fidelity recordings at 44.1 kHz, 16-bit mono, and smartphone recordings at 16 kHz, 16-bit mono. The Schubert Superlux ECM999 test microphone was utilized.
Item · Technical Parameters
Diaphragm Type · Back Electret
Directivity · Non-directional
Frequency Response · 20–20,000 Hz
Sensitivity · −37 dBV/Pa (14 mV)
Impedance · 200 Ω
Load Impedance · 1,000 Ω
Signal-to-Noise Ratio · 70 dB equivalent noise SPL: 24 dB
Sound Pressure Level · SPL: 132 dB
Dynamic Range · 108 dB
Dynamic Range · Analog: 100 dB, Digital: 104 dB
Item · Technical Parameters
Channel Count · 4
Data Format · WAV/BWF
Sampling Precision · 16/24-bit
Sampling Frequency · 44.1 kHz/48 kHz/88.2 kHz/96 kHz/192 kHz
Input Impedance · XLR: 4 kΩ or higher (balanced), TRS: 6 kΩ or higher (balanced)
Input Sensitivity · +24 dBu (input sensitivity knob: +4 dBu)
Output Impedance · Line Output: 600 Ω
Noise · Line Output: −100 dBu (input sensitivity: +4 dBu, input volume: medium)
Frequency Response · 20 Hz–40 kHz (0/-3 dB)
Dynamic Range · Analog: 100 dB, Digital: 104 dB
Mobile System · Phone Model
Android · Samsung NOTE 4, Samsung S6, OPPO A33, OPPO A59s, Huawei Honor Play 6X
Recording Method
Five recording positions were set up inside the vehicle, including two close-range positions and three functional positions. Functional positions were fixed directly in front of the driver, on the rearview mirror, and on the center console; close-range positions were located 20 cm away from the speaker, equipped with one high-fidelity microphone and one smartphone. Speakers read aloud the provided transcripts at normal speaking volume and pace.
Number · Location · Description
A · Driver’s Seat (Passenger-Side Simulation) Directly in Front · High-Fidelity Microphone
B · Roof of the Vehicle · High-Fidelity Microphone
C · Center Console · High-Fidelity Microphone
D · Rearview Mirror · High-Fidelity Microphone
Front of Speaker with Smartphone
5. Annotation and Transcription Guidelines
Transcribers are tasked with accurately rendering spoken content based on audio recordings, striving for complete alignment between written text and spoken utterances. Key principles include: 1) Transcriptions must precisely match the audio; no extra words, omissions, or errors are permitted. 2) Numerals should be rendered in Chinese characters, such as “一二三,” rather than “123.” Distinguish between “一” and “幺,” “二” and “两.” 3) English words appearing in sentences should be spelled out phonetically, e.g., “thank you.” Letters pronounced individually should be capitalized and separated by spaces, such as “N B A” or “U F O.” 4) Symbols within sentences should be transcribed according to actual pronunciation, e.g., “三W点百度点com.” Unpronounced symbols should be omitted. Brand names and proper nouns should follow standard conventions, such as “QQ空间,” “iPhone,” or “喜马拉雅.” 5) Transcriptions must fully reflect the original speech without any deletions.
6. Speaker Information
Basic Records
Speaker information includes task number, gender, accent region, age range, and place of origin.
Task Number · Gender · Accent Region · Age Range · Recording Scenario
1 · Male · Northern Accent · Scenario A · Scenario One
Each speaker receives a unique task number corresponding to a specific transcription assignment. Participation is limited to a single recording session. Gender: male or female. Accent regions: classified as Northern, Southern, or Other, based on native linguistic origins. Age ranges: A (16–25 years), B (26–40 years), C (41+ years). Recording Scenarios:
Speaker Demographic Characteristics
Gender Ratio
The database comprises 300 speakers, with 151 males and 149 females.
Gender · Male · Female · Total
Ratio · 50% · 50% · 100%
Age Distribution
Age group A (16–25): 300 individuals; B (26–40): 6 individuals; C (41+): 0 individuals.
Age Group · Age Bracket · Number of Individuals · Proportion · Male · Female
A · 16–25 years · 296 individuals · 98.7% · 149 individuals · 147 individuals
B · 26–40 years · 4 individuals · 1.3% · 2 individuals · 2 individuals
C · 41+ · 0 individuals · 0% · 0%
Total · 300 individuals · 100% · 151 males · 149 females
Dialect Proportion
Northern dialect: 294 individuals; Southern dialect: 8 individuals; Other dialect: 2 individuals.
Region · Number of Individuals · Proportion
Northern Region · 294 individuals · 98.0%
Southern Region · 8 individuals · 2.7%
Other Region · 2 individuals · 0.7%
Total · 300 individuals · 100%
Sample Data Download
Samples
- Audio sample 1