Segmentation fault (core dumped)
I install the whole pip packages and fix all env prombles, but error in decoding , the whole log is :
[NeMo I 2025-07-10 15:58:29 mixins:181] Tokenizer SentencePieceTokenizer initialized with 1024 tokens
[NeMo W 2025-07-10 15:58:29 modelPT:176] If you intend to do training or fine-tuning, please call the ModelPT.setup_training_data() method and provide a valid configuration file to setup the train data loader.
Train config :
use_lhotse: true
skip_missing_manifest_entries: true
input_cfg: null
tarred_audio_filepaths: null
manifest_filepath: null
sample_rate: 16000
shuffle: true
num_workers: 2
pin_memory: true
max_duration: 40.0
min_duration: 0.1
text_field: answer
batch_duration: null
use_bucketing: true
bucket_duration_bins: null
bucket_batch_size: null
num_buckets: 30
bucket_buffer_size: 20000
shuffle_buffer_size: 10000
[NeMo W 2025-07-10 15:58:29 modelPT:183] If you intend to do validation, please call the ModelPT.setup_validation_data() or ModelPT.setup_multiple_validation_data() method and provide a valid configuration file to setup the validation data loader(s).
Validation config :
use_lhotse: true
manifest_filepath: null
sample_rate: 16000
batch_size: 16
shuffle: false
max_duration: 40.0
min_duration: 0.1
num_workers: 2
pin_memory: true
text_field: answer
[NeMo I 2025-07-10 15:58:29 features:305] PADDING: 0
[NeMo I 2025-07-10 15:58:32 rnnt_models:226] Using RNNT Loss : tdt
Loss tdt_kwargs: {'fastemit_lambda': 0.0, 'clamp': -1.0, 'durations': [0, 1, 2, 3, 4], 'sigma': 0.02, 'omega': 0.1}
[NeMo I 2025-07-10 15:58:32 rnnt_models:226] Using RNNT Loss : tdt
Loss tdt_kwargs: {'fastemit_lambda': 0.0, 'clamp': -1.0, 'durations': [0, 1, 2, 3, 4], 'sigma': 0.02, 'omega': 0.1}
[NeMo W 2025-07-10 15:58:32 tdt_loop_labels_computer:281] No conditional node support for Cuda.
Cuda graphs with while loops are disabled, decoding speed will be slower
Reason: Driver supports cuda toolkit version 12.2, but the driver needs to support at least 12,3. Please update your cuda driver.
[NeMo I 2025-07-10 15:58:32 rnnt_models:226] Using RNNT Loss : tdt
Loss tdt_kwargs: {'fastemit_lambda': 0.0, 'clamp': -1.0, 'durations': [0, 1, 2, 3, 4], 'sigma': 0.02, 'omega': 0.1}
[NeMo W 2025-07-10 15:58:32 tdt_loop_labels_computer:281] No conditional node support for Cuda.
Cuda graphs with while loops are disabled, decoding speed will be slower
Reason: Driver supports cuda toolkit version 12.2, but the driver needs to support at least 12,3. Please update your cuda driver.
[NeMo I 2025-07-10 15:58:34 save_restore_connector:275] Model EncDecRNNTBPEModel was successfully restored from /root/.cache/huggingface/hub/models--nvidia--parakeet-tdt-0.6b-v2/snapshots/d97f7ac5d85e7185b7a7c4771c883c0e26d1d16f/parakeet-tdt-0.6b-v2.nemo.
Transcribing: 0%| | 0/1 [00:00<?, ?it/s]Segmentation fault (core dumped)
I don't know how to check and solve it, plz...
Hi, I also encountered a segmentation fault, but I was able to resolve it with this setting.
NeMo TDT Environment Setup
System Information
- GPU: NVIDIA A10G (22.1 GB VRAM)
- Driver Version: 580.95.05
- CUDA Version: 11.8
- OS: Ubuntu 24.04 LTS
- Python: 3.10.19 (conda environment)
Installed Components
Core Dependencies
- NVIDIA Driver: 580.95.05 (via driver 550 metapackage)
- CUDA Toolkit: 11.8.89
- Miniconda: Latest (Python 3.13 base)
- Conda Environment:
nemo_tdt(Python 3.10.19)
Python Packages
- PyTorch: 2.5.1 with CUDA 11.8 support
- Numba: 0.57.1 (FP16 support for TDT)
- NeMo Toolkit: 2.5.0
- PyTorch Lightning: 2.0.7
- Lightning: 2.4.0
- Transformers: 4.53.3
- Huggingface Hub: 0.23.0 (downgraded for NeMo compatibility)
- cuda-python: 11.8.3 (CRITICAL: Must match CUDA 11.8)