Skip to content

Quacomm AI Engine Direct - Enable FCB feature to Qualcomm HTP - #22521

Draft
quic-boyuc wants to merge 1 commit into
pytorch:mainfrom
CodeLinaro:dev1/boyuc/fcb-draft
Draft

Quacomm AI Engine Direct - Enable FCB feature to Qualcomm HTP#22521
quic-boyuc wants to merge 1 commit into
pytorch:mainfrom
CodeLinaro:dev1/boyuc/fcb-draft

Conversation

@quic-boyuc

Copy link
Copy Markdown
Contributor

QNN Flexible Context Binary (FCB) multi-SoC HTP support

Adds offline FCB export for shipping one ExecuTorch artifact containing QNN HTP
contexts for multiple known Snapdragon SoCs. Requires QNN SDK >= 2.48.

User-facing API

  • generate_qnn_executorch_compiler_spec() now accepts paired lists:
    soc_model=[...] and backend_options=[...].
  • Supplying two or more distinct HTP targets enables FCB; scalar inputs retain
    the existing single-SoC behavior.
  • Adds fcb_reference_weight_sharing=True by default. It shares referenced
    weights while host AOT appends target contexts to the FCB DLC.
  • QnnQuantizer and make_quantizer() accept FCB target lists and validate
    against the least-capable requested HTP architecture.
  • FCB rejects online preparation, non-HTP targets, mismatched target/options
    lists, duplicate SoCs, and DLBC when reference-weight sharing is enabled.

Architecture

  • Extends the QNN compiler-spec schema with paired FCB target records and
    reference-weight-sharing configuration.
  • Creates and caches one QNN manager per (backend type, SoC) during lowering.
  • Validates operator support against every requested target.
  • Reuses the same AOT op-wrapper graph sequentially for each target; graph
    registration state is reset after every compile, including failures.
  • Adds Python adaptor operations to create an FCB DLC, append each compiled
    context, retrieve its bytes, and free the DLC.
  • Uses QNN system DLC APIs at runtime to locate the compatible HTP context and
    deserialize its graph metadata.
  • Adds compiler-spec and manager-lifecycle coverage plus FCB ResNet50 and
    multi-SoC weight-sharing examples.

Usage

Build the Android runner:

 # Build with QNN SDK >= 2.48.
./backends/qualcomm/scripts/build.sh


# Measure FCB and HTP weight-sharing combinations across multiple targets:

python -m examples.qualcomm.util_scripts.fcb_multi_soc_weight_sharing_demo \
  --soc_models SM8650 SM8750 SM8850 SM8550 \
  --devices adb-host-a:<serial-a> adb-host-b:<serial-b> \
  --build_folder build-android

#  Export and optionally validate a floating-point FCB artifact:

python -m examples.qualcomm.util_scripts.fcb_resnet50 \
  --soc_models SM8650 SM8750 \
  --devices <serial-on-SM8650> <serial-on-SM8750> \
  --build_folder build-android

# For devices attached to separate ADB servers, use host:serial values:

python -m examples.qualcomm.util_scripts.fcb_resnet50 \
  --soc_models SM8650 SM8750 \
  --devices adb-host-a:<serial-a> adb-host-b:<serial-b> \
  --build_folder build-android

# run FCB with quantized reset50   

python -m examples.qualcomm.util_scripts.fcb_resnet50_quantized \
     --soc_models SM8550 SM8850 SM8750 SM8650 \
     --dataset ~/executorch/imagenet-mini-val/val \
     --output_dir /tmp/qnn_fcb_resnet50_quantized \
     --devices adb-host-a:<serial-a> adb-host-b:<serial-b> \
     --build_folder build-android

@pytorch-bot

pytorch-bot Bot commented Sep 3, 2026