AudioMandarin Chinese

Smart Cockpit Voice Interaction Dataset

About this dataset

1. Data Overview

The intelligent cockpit voice interaction dataset comprises a total of 3,164 hours. The recordings are in Chinese and were collected in China, involving 260 speakers aged between 7 and 55 years old. All recordings were made under real-world conditions inside the Li Auto ONE vehicle. The dataset includes three categories: reading (561 hours), dialogue (2,163 hours), and in-vehicle noise (440 hours). Recording equipment covers three types, with a total of 11 recording locations: close-range head-mounted microphone (48 kHz, 16-bit, 210 hours), in-car infotainment system (16 kHz, 16-bit, 824 hours, four microphones), overhead seat microphone (16 kHz, 16-bit, 846 hours, four microphones), roof-mounted microphone (16 kHz, 16-bit, 424 hours), and central control screen microphone (16 kHz, 16-bit, 421 hours). This database has been manually verified, achieving a text accuracy rate exceeding 95%, making it suitable for research in speech recognition, speaker verification, and related fields.

2. Recording Corpus

Corpus Preparation

Design of Recorded Texts

The dataset is divided into reading and dialogue sections. For closed-window scenarios, each speaker recorded 50 sentences; for open-window scenarios, 100 sentences per person. The corpus covers areas such as navigation, media, and more. Dialogue sessions involve groups of 2–4 participants, with two primary speakers. Each session lasts approximately 10 minutes, focusing on casual conversation and free exchange. Specific subcategories within reading and dialogue are as follows: Reading: commands related to vehicle controls, such as air conditioning and seats; Media: playback, music search, radio listening; Contacts: making calls, querying contacts; Navigation: POI inquiries; Knowledge Q&A and voice search; Dialogue: human–machine casual chat and free in-car conversations.

Corpus Processing

De-identification: Removal of politically sensitive, personally private, or sexually explicit content.
Removal of symbols such as <, >, [, ], ~, /, \, =, except for numeric strings.
Exclusion of content containing languages other than Chinese and English.
Standardization of format.

3. Speaker Information

Speaker Demographic Characteristics

Gender Ratio

The dataset includes a total of 260 speakers—118 males and 142 females.

Gender · Male · Female · Total

Proportion · 45.4% · 54.6% · 100%

Age Distribution

The age range of speakers spans from 7 to 55 years old, distributed as follows: A (7–17 years); B (18–25 years); C (26–40 years); D (41–55 years).

· Age Group · Number of Speakers · Proportion

A · 7–17 years · 25 · 9.6%

B · 18–25 years · 122 · 46.9%

C · 26–40 years · 47 · 18.1%

D · 41–55 years · 66 · 25.4%

Total · 260 · 100% ·

4. Recording Environment

Recording Conditions

All recordings were conducted in real in-vehicle settings, capturing both daytime and nighttime scenarios. Participants recorded under various conditions, including parking, driving, and opening/closing windows. Specific scenarios are detailed below:

Window Opening/Closing Scenarios ·

No. · Scenario Description

1 · Parking, air conditioning off, AEC off

2 · Parking, air conditioning at medium setting, AEC off

3 · Parking, air conditioning at high setting, AEC at medium setting

4 · Non-highway driving (0–60 km/h), air conditioning off, AEC off

5 · Non-highway driving (0–60 km/h), air conditioning at medium setting, AEC off

6 · Non-highway driving (0–60 km/h), air conditioning at high setting, AEC at medium setting

7 · Low-speed driving (0–40 km/h), air conditioning off, AEC off

8 · Low-speed driving (0–40 km/h), air conditioning at medium setting, AEC off

9 · Low-speed driving (0–40 km/h), air conditioning at high setting, AEC at medium setting

10 · Medium-speed driving (40–80 km/h), air conditioning off, AEC off

11 · Medium-speed driving (40–80 km/h), air conditioning at medium setting, AEC off

12 · Medium-speed driving (40–80 km/h), air conditioning at high setting, AEC at medium setting

13 · High-speed driving (80–120 km/h), air conditioning off, AEC off

14 · High-speed driving (80–120 km/h), air conditioning at medium setting, AEC off

15 · High-speed driving (80–120 km/h), air conditioning at high setting, AEC at medium setting

Recording Equipment

The recording setup includes head-mounted microphones, in-cabin microphones, linear microphone arrays spaced 5 cm apart, and single-microphone setups.

Recording Method

A total of 11 recording locations were used, covering close-range speaking positions, overhead seats, central control screens, and the vehicle roof. The following diagram illustrates the placement of recording devices:

Dialogue sessions were recorded using head-mounted microphones, with the recording position adjusted according to predefined rules to ensure coverage of all in-vehicle audio pickup zones.

5. Speech Transcription

Transcribers convert audio recordings into written text, striving to maintain consistency between spoken words and written content. General guidelines include: 1) Transcriptions must exactly match the spoken audio, without adding, omitting, or misrepresenting any words. 2) Numbers should be transcribed in Chinese characters—for example, “一二三” instead of “123.” Pay special attention to distinguishing between “一” and “幺,” and between “二” and “两.” 3) Any English pronunciation in the audio should be rendered in corresponding Chinese characters or left as English. Specific cases include: All letters or words in URLs should be capitalized—for instance, “www.abc.com” should be transcribed as “三W点A B C点com.” English words in the audio should be transcribed entirely in lowercase. English letters in the audio should be transcribed entirely in uppercase. Certain proper nouns or acronyms requiring capitalization should retain their original form, such as “C E O” or “C C T V.” 4) The completeness of annotations must align with actual pronunciation, with no omissions allowed.

6. Data File Directory

Directory Structure

The data directory tree is organized as follows:

└─DOC（Textual Documentation）
││├─spkrinfo.xlsx（Speaker Information）
│
└─SPEECHDATA（Data Folder）
│├─M (Closed Window Scenario)
│├─A1（Recording Scene）
│├─dialogue（Recording Format）
│├─030（Dialogue Number）
││ ││ DA02.wav（Audio File）
││ ││ DA02.TextGrid（Transcribed Text）
││ ││ DL01C01.wav（Audio File）
││ ││ DL01C01.TextGrid（Transcribed Text）
││ ││ DX01C01.wav（Audio File）
││ ││ DX01C01.TextGrid（Transcribed Text）

……

│├─reading（Recording Format）
│├─Audio（Audio File）

│├─P0059（Speaker ID）
││ ││ P0059_A02_0001.wav（Audio File）
│├─Label（Text Folder）
│├─P0059.txt（Transcribed Text）
│├─N1（Open Window Scenario）

……

│├─N2（Open Window Scenario）

…..

│├─N3 (Open Window Scenario)

……

Naming Conventions

Directory Naming Rules

<USAGE>/<Window_info>/<Scene_ID>/<Recording_info>/<Dialogue_ID>/<wav_ID>e.g. SPEECHDATA/M/A1/dialogue/030/DA02.wav

Directory · Content · Notes

USAGE · SPEECHDATA · Folder Name

Window_info · M/N1/N2/N3 · Window Opening/Closing Scenarios

Scene_ID · A~L · Recording Scene Number

Recording_info · dialogue · Dialogue Recording

Dialogue_ID · 0030 · Dialogue Number

wav_ID · DA02.wav · Audio File

<USAGE>/<Window_info>/<Scene_ID>/<Recording_info>/<File_type>/<Speaker_ID>/<wav_ID>e.g. SPEECHDATA/M/A1/reading/Audio/P0059/P0059_A02_0001.wav

Directory · Content · Notes

USAGE · SPEECHDATA · Folder Name

Window_info · M/N1/N2/N3 · Window Opening/Closing Scenarios

Scene_ID · A~L · Recording Scene Number

Recording_info · Reading · Reading Recording

File_type · Audio/Label · File Type

Speaker_ID · P0059 · Speaker ID

wav_ID · P0059_A02_0001.wav · Audio File

序号 · Number · Notes

1 · A01~04、DA01~04 · 舒尔头戴、D开头代表是对话

2 · X01~06 · 车机，05和06通道是回采信号

3 · L01C01~C02 · 中控屏线阵，间距50mm

4 · L02C01~C02 · 车顶线阵，间距50mm

5 · S01~04 · 座位上方单麦

Equipment Naming Rules

The recording process involves three types of equipment, totaling seven recording locations. Equipment numbers are listed below:

Scene Naming Rules

Scenes encompass window opening/closing, vehicle speed, air conditioning settings, and other factors. Specific scene numbers are provided in the table below:

N: Open Window Scenarios（Three open window scenarios share the same number） ·

Scene Number · Scene Description

A · Parking, air conditioning off, AEC off

B · Parking, air conditioning at medium setting, AEC off

C · Parking, air conditioning at high setting, AEC at medium setting

D · Non-highway driving (0–60 km/h), air conditioning off, AEC off

E · Non-highway driving (0–60 km/h), air conditioning at medium setting, AEC off

F · Non-highway driving (0–60 km/h), air conditioning at high setting, AEC at medium setting

N1: Main driver opens one-third of the car window, sunroof remains closed. N2: Car window closed, sunroof opened halfway. N3: Main driver opens half of the car window, sunroof opened halfway.

M: Closed Window Scenarios ·