Each paper extends a vision-language-action model (or a VLM-based policy) with an extra sensing input. Grouped by modality; within a group, by arXiv date. “Vision-only → with modality” is the paper’s own ablation where one is reported. Depth and point-cloud VLAs are out of scope.
Tactile (29)
+
01
Peng Hao … Shuo Wang · Samsung Research China / CASIA · arXiv Mar 2025 · arXiv:2503.08548
GelStereo 2.0 ×2 (tactile only, no RGB) · backbone Qwen2-VL + Qwen2-7B (LoRA) · 8 tactile frames tiled into one image → ViT → tokens in LLM; integer action tokens · TLA 88–90% vs Diffusion Policy 40% (no vision-only baseline)
02
Chaofan Zhang … Shuo Wang · CASIA / Samsung / BAAI · arXiv May 2025 · arXiv:2505.09577
GelStereo 2.0 ×2 (sim→real UR3) · backbone Qwen2-VL-7B · temporally-enhanced tactile tokens before visual tokens; next-token actions + DPO · sim 0.6 mm peg 80%→90%; real triangle 90%→95%
03
Jialei Huang … Yang Gao · Tsinghua · arXiv Jul 2025 · arXiv:2507.09160
dual high-res normal+shear tactile on handheld UMI gripper · backbone π0 · MLP over tactile history → 1 token in VL prefix; force in action space + hybrid position/force control · USB 5%→35%; charger 40%→90%
04
Jianxin Bi … Harold Soh · NUS · RA-L 2026 · arXiv:2507.17294
GelSight Mini · backbone RDT-1B (frozen) + Octopi + GPT-4o · planning-time tactile captions + diffusion controller refining action chunks · cup 35→50%, wipe 25→60%, peel 30→50%
05
Zhengxue Cheng … Li Song · SJTU + Paxini · arXiv Aug 2025 · arXiv:2508.08706
Paxini Gen2 Hall-effect force arrays · backbone π0 · force maps stitched into image → SA-ViT (contrastive, ObjTac) → 256 tokens · gripper 75.0→96.9%; peg 50→83.3%; dex-hand 93.8→100%
06
Yuzhe Huang … Ziyuan Jiao · Beihang / ShanghaiTech / BIGAI / HKU · arXiv Jan 2026 · arXiv:2601.20321
GelSight Mini + custom vision-based tactile · backbone π0.5 · TaF-Adapter aligned to VQ force codes (6-axis wrench + pressure map) conditions flow expert · +22% avg over aligned baseline
07
Guo Ye … Han Liu · Northwestern · arXiv Dec 2025 · arXiv:2512.23864
GelSight ×2 · backbone CLIP ViT-L/14 + ACT-style expert (borderline: no pretrained VLA) · frozen V-JEPA2 tactile encoder + adapter; future-tactile forecast fed back · peg 35.2→45.6%; gear 22.4→40.2%
08
Kaidi Zhang … Yu She · Purdue · arXiv Mar 2026 · arXiv:2603.12665
15×8 tactile array (one finger) · backbone π0.5 (LoRA) · MLP → 36 tactile tokens; contact flag masks tokens when no contact · disassembly 63.75→83.75%; in-box picking 10→70%
09
Charlotte Morissette … Gregory Dudek · McGill / Mila · ECCV 2026 · arXiv:2603.14604
DIGIT · backbone OpenVLA-OFT · FiLM: Sparsh-DINO tactile embedding modulates ViT blocks · 58.1→86.67%
10
Konstantin Gubernatorov … Dzmitry Tsetserukou · Skoltech · arXiv Mar 2026 · arXiv:2603.15257
100-taxel/finger arrays (training only) · backbone SmolVLA · reward-weighted flow matching teacher + tactile-token distillation to sensor-free student · 41.7→86.7%
11
Xiaoqi Li … Hao Dong · PKU / PrimeBot · arXiv May 2026 · arXiv:2605.07308
Xense 3-D force tactile · backbone GO-1 (InternVL-2B + DiT) · learned contact gate; tactile token replaces state token as cross-attn query · 0.22→0.50 avg
12
Huaihang Zheng … Baoxu Liu · Meituan / BIT / Beihang / CASIA · arXiv Jun 2026 · arXiv:2606.09337
Tacta 6×8 pads → 6-DoF fingertip wrench · backbone π0.5 · wrench-history token + soft MoE + zero-init bypass; online RL · 12/30→28/30
13
Siyu Ma … Chenfanfu Jiang · UCLA / UCSD · arXiv Jun 2026 · arXiv:2606.11743
FlexiTac-V2 32×12 pads · backbone π0.5-style · tactile tokens via cross-attn + contact gate; sim-real co-training + sim RL · 50.0→72.5% (4 bimanual tasks)
14
Chengbo Yuan … Yang Gao · Tsinghua / Qi Zhi / Sharpa · arXiv Jun 2026 · arXiv:2606.13102
21 sensor types in pretraining (GelSight Mini, Sharpa DTC, Xense, Contactile…) · backbone π0.5 · sensor-specific encoders → morphology-aware token space → 300M tactile expert · seen 45.3→62.5%; unseen sensors 15.0→46.6%
15
Dantong Niu … Trevor Darrell · UC Berkeley / NVIDIA / Stanford · arXiv Jun 2026 · arXiv:2606.17055
Sharpa Wave per-finger force + deformation · backbone MoT VLA (latent/action/tactile experts) · VQ-VAE force history + conv deformation → tactile expert refining actions at high rate · 42→65% (6-task ablation)
16
Mark Van der Merwe … Nima Fazeli · U Michigan · arXiv Jun 2026 · arXiv:2606.29089
GelSight ×2 · backbone π0.5 · shear fields drawn onto RGB views; architecture unchanged · <50% → 78% (94/120)
17
Xidong Zhang … Weihao Yuan · HIT / Great Bay / Daimon / NJU · arXiv Jun 2026 · arXiv:2606.31723
DM-Tac W ×2 · backbone π0.5 · unified tactile tokens + tactile chain-of-thought + future-tactile prediction + residual controller · USB 18→62%; pencil 40→86%
18
Pengwei Zhang … Tiancai Wang · UCAS / CASIA / Dexmal · arXiv Jul 2026 · arXiv:2607.03387
GelSight Mini (one fingertip) · backbone π0.5 · tactile residual vs vision-predicted latent → VQ contact primitives, surprise gate · 28.2→62.8% avg (5 tasks)
19
Ruilin Chen … Wenqiang Zhang · Fudan / Lenovo / NTU / TeleAI · arXiv Jul 2026 · arXiv:2607.14609
PaXini tactile on ARX R5 · backbone SmolVLA (also π0) · training-only latent tactile predictor from action-expert features; dropped at inference · SmolVLA 18→74%; π0 40→73%
20
NeoteAI & Fudan TEAI · arXiv Jul 2026 · arXiv:2607.23782
self-developed vision-based tactile · backbone PaliGemma + flow expert (π0-like) · predictor emits latent tactile tokens conditioning action expert · 63.8% vs 44.0% best baseline (20-task sim)
21
Ning Cheng … Wenjuan Han · Beijing Jiaotong / BIGAI · arXiv Jul 2026 · arXiv:2607.24485
DM-Tac WS ×2 · backbone π0.5 · touch encoder + adapter tokens; JEPA-style future-visual-latent loss · 28.75→71.25% (4 tasks)
22
Shiqi Zhang … Yanyong Zhang · USTC / iFLYTEK / CUHK · arXiv Aug 2026 · arXiv:2608.01824
XHand 120 3-axis taxels/finger · backbone unnamed VLM + Foresight Action Expert (baselines π0.5) · tactile-patch tokens, foresight queries, online refinement · π0.5 51.5% → 83.6%
23
Yi Wang … Xuesong Li · BIT · arXiv Aug 2026 · arXiv:2608.15816
9DTact (real), sim GelSight Mini · backbone frozen OpenVLA-OFT · tactile-gated bounded residual corrections · sim 30.7→61.3%; real ≈18→48.3%
24
NeoteAI & Fudan TEAI · arXiv Aug 2026 · arXiv:2608.29601
camera-based visuo-tactile → NeoForce; N0-TacUMI handheld · backbone shared transformer, π0.5-class · tactile patches tokenized with RGB; force-field reconstruction + latent prediction · 26.5→32.5% (Tactile Gap row)
25
Yankai Fu … Shanghang Zhang · PKU / BAAI · arXiv Sep 2026 · arXiv:2609.09119
Sharpa fingertip VBTS on SharpaWave hands · backbone Mixture-of-Transformers (Qwen3-VL + Qwen3 experts) · adaptive visuo-tactile cross-attn with contact gate; latent co-imagination · ≈33% (InternVLA-A1) / ≈32% (GR00T N1.6) → 71%
26
Xiangcheng Liu … Jianlan Luo · Shanghai Innovation Institute · arXiv Sep 2026 · arXiv:2609.12549
268-d piezoresistive arrays, 10-DoF hands · backbone π0.5 · tactile rendered as image → patch tokens; contact gate; sparse future-tactile loss · 44→61% (4-task)
27
Zhengyu Tao … Xin Wang · Texas A&M · arXiv Sep 2026 · arXiv:2609.20980
TSF-85 7×4 arrays on Robotiq 2F-85 · backbone π0.5 (LoRA) · bidirectional cross-attn; transformer forecasts tactile +5…+20 steps · 58.75→95.0% (4 tasks)
28
Chen Yang … Chen Wang · Tsinghua / Alibaba / HK PolyU · arXiv Oct 2026 · arXiv:2610.02784
vision-based tactile, parallel gripper · backbone π0.5 · frozen T3 encoder tokens → tactile expert attended layer-wise; multi-horizon future-tactile loss · sim 77.5% vs FTP-1 66.7%; real 71.3 vs 62.5%
29
Qiwei Wu … Renjing Xu · NVIDIA Beijing / HKUST-GZ · arXiv May 2026 · arXiv:2605.27886
simulated GelSight-style (Isaac Lab) · backbone π0 · tactile tokens; outputs pose + fingertip force setpoints (action-space force) · 0.00→0.86 (firm); grip force −70% (gentle)
Force, torque and wrench (19)
+
30
Jiawen Yu … Cewu Lu · Fudan / SJTU / SII / Noematrix · NeurIPS 2025 · arXiv:2505.22159
6-axis F/T (Flexiv estimated wrench) · backbone π0 · FVLMoE: force token → self-attn + sparse MoE → added to action tokens · 37.3→60.5% (naive concat 40.2%)
31
Zongzheng Zhang … Hao Zhao · BAAI / Tsinghua AIR / NTU · arXiv Sep 2025 · arXiv:2509.07962
joint torque from motor current (no F/T sensor) · backbone π0 (also RDT, ACT) · torque-history adapter token in decoder + auxiliary torque prediction · button 5/20→18/20; charger 0→17; USB 0→17; door 2→15
32
Ruiteng Zhao … Haiyue Zhu · NUS / A*STAR · ICRA 2026 · arXiv:2602.02142
UR5e internal force (training only) · backbone SmolVLA · learnable query → predicted force token aligned to measured force; sensor-free at inference · 23.3→61.1% (raw-force 38.9%)
33
Yao Li … Yanyong Zhang · USTC · arXiv Mar 2026 · arXiv:2602.23648
6-axis F/T at flange · backbone π0 · TCN force tokenizer; cross-attn from action tokens every layer; history tokens in prefix · ≈42.8→80.8% (ForceVLA ≈67.0%)
34
Yang Li … Jiangmiao Pang · Shanghai AI Lab · CVPR 2026 · arXiv:2603.15169
6-D F/T 300 Hz, Flexiv Rizon 4s · backbone π0 · text force prompts + force embedding → cross-scale MoE; outputs pose + force · 17→66% avg
35
Ruicheng Li … Baining Guo · Tsinghua / Microsoft Research · arXiv Jul 2026 · arXiv:2607.18231
wrist 6-axis F/T 100 Hz, AgiBot G1 · backbone π0.5 · VAE-compressed wrench history → 8 force tokens after action tokens; + joint-state-history token · 27.8→83.3%
36
Yi Wang … Cewu Lu · SJTU / SII / Noematrix · arXiv Jul 2026 · arXiv:2607.14236
Flexiv built-in 6-D F/T · backbone π0.5 · grafted reactive action expert, GRU force memory via zero-init cross-attn; online DAgger · book 0.4→0.6; Hanoi 0.3→0.6; towel 0.725→0.825
37
Carlota Parés-Morlans … Jeannette Bohg · Stanford · arXiv Aug 2026 · arXiv:2608.01402
Bota SensONE wrist F/T 400 Hz, FR3 · backbone π0.5 (LoRA; also π0) · pooled F/T → per-layer AdaRMS modulation; F/T history tokens; contact-gradient gating · 39.0→66.0% (ForceVLA 40.5%, TA-VLA 37.5%)
38
Xingzheng Wu … Qing Cai · arXiv Aug 2026 · arXiv:2608.15009
force + ultrasound images · backbone unnamed · Force-Ultrasound fusion + stage-adaptive modulation
39
Yutian Zhang … Dibo Hou · Zhejiang U / Shanghai AI Lab / Tsinghua / Deep Robotics · arXiv Sep 2026 · arXiv:2609.03889
sensorless residual-torque contact estimator (wheeled-legged) · backbone π0.5 · GRU force tokens → gated cross-attn into action expert
40
Zhaohong Mai … Chenguang Yang · SCUT / Ola Dimensions / HK PolyU / Huawei · arXiv Sep 2026 · arXiv:2609.05832
6-DoF F/T on UR5e · backbone π0 · force-history token + sparse MoE residual; adaptive compliance controller · π0 47.3% (ForceVLA 73.2%) → 89.2%
41
Ju Dong … Jianwei Zhang · U Hamburg · arXiv Sep 2026 · arXiv:2609.18242
joint-torque wrench estimates · backbone ForceVLA (π0) teacher + student · distilled force-correction + delay-correction student between chunk updates · ForceVLA 54.4→82.2% (9 tasks); −26% peak force
42
Bohan Gan … Ji Ge · China Mobile · arXiv Sep 2026 · arXiv:2609.18497
wrist 6-axis F/T, AgiBot G1 · backbone GR00T N1.5 (frozen) · F-FiLM zero-init residual on VL features; modality dropout; fast admittance branch · 80.0% ID / 64.2% OOD
43
Fukang Liu … Ye Zhao · Georgia Tech · arXiv Sep 2026 · arXiv:2609.23968
joint-torque history (in) + contact-force references (out); Agility Digit · backbone GR00T N1.7 · torque history input; action space extended with force references tracked by RL controllers · 40.0→57.8%; force MAE 11.2→4.2 N
44
Pankhuri Vanjani … Rudolf Lioutikov · KIT · arXiv Jun 2026 · arXiv:2606.12105
Franka torque-estimated F/T + proprio at 100 Hz · backbone X-VLA · per-modality latent buffers at native rates; gated cross-attn in action expert · 40.95→95.2% (7 tasks)
45
Jiaqi Zhai … Wei Zhang · arXiv Sep 2026 · arXiv:2609.15012
contact history (sensor unnamed) · backbone unnamed VLA · codebook-aligned coordinate in action-expert blocks; bounded residual · plug/vase 59.0/71.5→78.5/75.2
46
Jung-Woo Lee … Soo-Chul Lim · Dongguk · arXiv Sep 2026 · arXiv:2609.25785
fingertip forces (RH56F1) rendered as visual cues · backbone VLA-based (unnamed) · forces drawn on images; goal-conditioned cross-attn · 40–80% per task; no ablation
47
Shuoheng Zhang … Jianye Hao · Tianjin U / Huawei / Shanghai AI Lab · arXiv May 2026 · arXiv:2605.11048
F/T sensing · backbone DiT flow policy + VLM pointing (borderline) · force history + proprio via AdaLN · ForceVLA 45→81.67%
48
Zijian An … Lifeng Zhou · arXiv Sep 2025 · arXiv:2509.14143
digital scale readout (weight) · backbone π0 + CLIP · planning-time text directives
Audio and speech (7)
+
49
Wei Zhao … Donglin Wang · Westlake U · ICLR 2025 · arXiv:2502.13508
microphone speech (instruction channel) · backbone LLaVA (CLIP + Vicuna) · Whisper mel features → projector into LLM; voice RAG · CALVIN speech 3.70 vs text 3.74; custom tasks 86.5 vs 19.2%
50
Siyin Wang … Chao Zhang · Tsinghua / ByteDance · arXiv Oct 2025 · arXiv:2510.16756
streaming speech · backbone Llama-3.1-8B speech expert + UniVLA action expert · 1-s audio blocks interleaved with frames · LIBERO ≥ text-conditioned
51
Xiaoyu Liu … Ran He · arXiv Oct 2025 · arXiv:2510.21817
user speech · backbone two parallel VLA instances · VLM emits control tokens
52
Siyin Wang … Xipeng Qiu · Fudan / SII / NUS · ICLR 2026 · arXiv:2510.23763
spoken dialogue + environmental sounds (raw audio) · backbone Qwen2.5-Omni-style omni-LLM; FAST+ actions · audio tokens fused with vision in omni-LLM · 85.6 vs 25.9% (NORA); real 76.6 vs π0 73.8%
53
Xiangyi Wei … Changbo Wang · ECNU · arXiv Nov 2025 · arXiv:2511.09958
2 piezo contact mics on gripper, 44.1 kHz · backbone OpenVLA-style (Llama-2 7B + DINOv2/SigLIP) · AudioCLIP spectrogram → tokens concat with visual/proprio · RLBench 48.0→55.1%; real 20→60%
54
Chang Nie … Hesheng Wang · SJTU / Cambridge · IJRR 2026 · arXiv:2603.16086
streaming environmental audio + proprio · backbone omni foundation model + flow-matching Realizer · causal audio memory + audio world model · HEAR-Bench 0.81 vs π0.5-Waveform 0.61; real 0.70 vs 0.45
55
Ziang Guo … Zufeng Zhang · IEEE IV 2026 · arXiv:2601.12142
spoken instructions incl. prosody · backbone Qwen2.5-Omni · synthetic speech-aligned nuScenes + emotional CoT · −59.4% L2, −74.4% collisions
EMG, gaze and other human biosignals (4)
+
56
Edward W. Staley … Michael Wolmetz · Johns Hopkins APL · arXiv 1 Oct 2026 · arXiv:2610.01794
8-ch Myo armband sEMG envelopes (forearm), 3 participants · backbone SmolVLA (frozen; action expert + state projection trained) · 8-d EMG envelope concatenated to 6-d proprioceptive state · in-dist 0.80 vs 0.80; cluttered OOD 0.30→0.50 (completion 0.43→0.82)
57
Kuangji Zuo … Jianfei Yang · NTU · arXiv May 2026 · arXiv:2605.30282
Meta Aria eye-tracking (gaze) · backbone GR00T N1.5 · gaze contours on robot image + gaze token via zero-init cross-attn · 23.2→83.5% (Unitree G1)
58
Yihan Zhou … Yilin Mo · arXiv Sep 2026 · arXiv:2609.34550
VR-teleop eye tracker (train); predicted gaze at inference · backbone π0 · gaze crosshair rendered on head-camera images · 26.3→56.0% (6 bimanual tasks)
59
Anupam Pani, Yanchao Yang · CVPRW 2026 · arXiv:2603.23202
human gaze heatmaps (training-time only) · backbone existing VLAs · KL regularization of attention to gaze · +4–12%
Thermal, haptic and proprioceptive (5)
+
60
Dian Yu … Zewen Yang · TU Munich · arXiv Mar 2026 · arXiv:2603.05754
Topdon TC001 LWIR thermal (+RGB-D) · backbone GR00T N1.5-3B · thermal pseudocolor as own image-token stream; CBF safety filter · hot/cold bottle 42→86%; night 0→64–72%
61
Young-Chae Son … Soo-Chul Lim · Dongguk · RA-L 2026 · arXiv:2603.25044
ThermoEye TMC256B thermal · backbone π0 (LoRA) + Gemini planner · thermal image replaces external RGB view · 59.4→76.8%; thermal subtasks ≈42→82%
62
Luis F. Moreno Fuentes … Dzmitry Tsetserukou · Skoltech · arXiv Aug 2025 · arXiv:2508.01361
haptic OUTPUT (mid-air cues), no haptic input · backbone OpenVLA (LoRA) · 7-D action = 3 velocity + 4 haptic commands · 56.7% target acquisition
63
Yiren Zhao … Rushi Dai · arXiv Aug 2026 · arXiv:2608.03052
proprio history 1–96 frames · backbone flow-matching VLA · compares 5 interfaces for state injection · design study
64
— · ACM MM 2026 · arXiv:2608.16074
ultrasound imaging · backbone — · — · unverified
Three or more sensing modalities (4)
+
65
Heyu Guo … Lili Qiu · Princeton / MSR Asia · arXiv Nov 2025 · arXiv:2511.01210
IR thermal + mmWave radar + 6-mic array · backbone SmolVLA (also π0) · sensor heatmaps blended into object masks → ‘sensor-masked image’ → per-sensor tokens · RGB-only 25→84% (3 sensor tasks)
66
Zhuoyang Liu … Shanghang Zhang · PKU · arXiv Sep 2025 · arXiv:2509.26642
RGB + point cloud + Tashan tactile (6 values/finger) · backbone LLaMA-2 7B (Prismatic init), encoder-free · tactile token at projected gripper position; contrastive alignment; multisensory forecasting · +12% vs π0, +24% vs SpatialVLA
67
Jimin Lee … Jinwoo Shin · KAIST / RLWRLD · arXiv Apr 2026 · arXiv:2604.23272
AnySkin tactile + joint torque + fingertip force · backbone GR00T N1.5 and π0 · one stream per signal mirroring the action expert; cross-modal attention; future-signal loss · GR00T 20.8→49.0%; π0 27.1→45.9%
68
Yining Hong … Chuang Gan · arXiv Jan 2024 · arXiv:2401.08577
visual + audio + tactile + thermal (simulated) · backbone pretrained LLM with action tokens · object-centric multisensory tokens · sim only
List B. EMG to hand pose:
Papers estimating continuous hand pose from surface EMG (116 papers)
Papers whose main contribution is predicting continuous finger or hand pose (joint angles, keypoints or mesh) from sEMG, 2018 – 12 October 2026. Sorted by year. Each entry notes the EMG placement, whether it is a strict multi-finger result or a borderline one, the community it comes from, and whether it builds on Meta’s emg2pose. Entries marked “title and venue only” could not be opened from here. Counts are lower bounds.
2018 (4)
+
01
Quivira … Erdogmus · MERL/Northeastern · IEEE BHI 2018 · doi:10.1109/BHI.2018.8333395
forearm (Myo 8ch) · multi-finger pose from sEMG alone · wristband / AR-VR / robotics
02
Dai, Cao, Hu · NC State · IEEE EMBC 2018 · doi:10.1109/EMBC.2018.8513152
forearm HD grid · borderline (≤3 DoF, sensor fusion, demo or hardware) · prosthetics / neural engineering
03
Melcer … Remaley · CTRL-labs · CHI 2018 EA demo · doi:10.1145/3170427.3186520
forearm 16ch band · borderline (≤3 DoF, sensor fusion, demo or hardware) · wristband / AR-VR / robotics
04
Pan, Crouch, Huang · NC State/UNC · IEEE TNSRE 2018 · doi:10.1109/TNSRE.2018.2838448
forearm 4ch · borderline (≤3 DoF, sensor fusion, demo or hardware) · prosthetics / neural engineering · title and venue only
2019 (5)
+
05
Chen, Yu, Sheng, Farina, Zhu · SJTU/Imperial · J Neural Eng 2019 · doi:10.1088/1741-2552/aaf4c3
forearm HD grid · multi-finger pose from sEMG alone · prosthetics / neural engineering · title and venue only
06
Stapornchaisit … Koike · Tokyo Tech · Front Neurorobot 2019 · doi:10.3389/fnbot.2019.00075
forearm 96-ch array · multi-finger pose from sEMG alone · prosthetics / neural engineering
07
Krasoulis, Vijayakumar, Nazarpour · Edinburgh/Newcastle · Front Neurosci 2019 · doi:10.3389/fnins.2019.00891
forearm 16ch (DB8) · multi-finger pose from sEMG alone · prosthetics / neural engineering
08
Matran-Fernandez … Citi · Essex/SSSA · Sci Data 2019 · doi:10.1038/s41597-019-0200-9
forearm 134ch HD · multi-finger pose from sEMG alone · prosthetics / neural engineering
09
Pan, Crouch, Huang · IEEE TNSRE 2019 · doi:10.1109/TNSRE.2019.2937929
forearm 4ch · borderline (≤3 DoF, sensor fusion, demo or hardware) · prosthetics / neural engineering
2020 (7)
+
10
Dai, Hu · IEEE JBHI 2020 · doi:10.1109/JBHI.2019.2926307
forearm HD 8×8 · borderline (≤3 DoF, sensor fusion, demo or hardware) · prosthetics / neural engineering
11
Dwivedi, Ngeo, Shibata · Kyutech · IEEE TBME 2020 · doi:10.1109/TBME.2020.2967154
forearm 8 muscles · multi-finger pose from sEMG alone · prosthetics / neural engineering
12
Batayneh, Abdulhay, Alothman · JUST · Heliyon 2020 · doi:10.1016/j.heliyon.2020.e03669
forearm · multi-finger pose from sEMG alone · prosthetics / neural engineering
13
Koch … Mertins · Lübeck · IEEE EMBC 2020 · doi:10.1109/EMBC44109.2020.9176278
NinaPro · multi-finger pose from sEMG alone · prosthetics / neural engineering
14
Blana … Chadwick · Keele/Newcastle · IEEE TNSRE 2020 · doi:10.1101/629246
EMG-driven MSK · multi-finger pose from sEMG alone · prosthetics / neural engineering · title and venue only
15
Estimation of finger joint angle based on sEMG using LSTM
Anam … Ismail · Univ. Jember · conf. 2020 · repository.unej.ac.id/…/103243
2× Myo · multi-finger pose from sEMG alone · prosthetics / neural engineering
16
Continuous finger joint angle estimation with sEMG signals
— · IEEE conf. 2020 · ieeexplore 9293963
— · multi-finger pose from sEMG alone · prosthetics / neural engineering · title and venue only
2021 (11)
+
17
Guo … Lin · Dalian Maritime/Imperial · J Neural Eng 2021 · doi:10.1088/1741-2552/abd461
NinaPro DB2 · multi-finger pose from sEMG alone · prosthetics / neural engineering
18
Chen … Lin · Appl Sci 2021 · doi:10.3390/app11104678
NinaPro DB2 · multi-finger pose from sEMG alone · prosthetics / neural engineering
19
Gao, Tang, Huang, He · Sensors 2021 · doi:10.3390/s21082576
NinaPro DB5 (2× Myo) · multi-finger pose from sEMG alone · prosthetics / neural engineering
20
Bao … Zhang · Leeds · Pattern Recognit Lett 2021 · doi:10.1016/j.patrec.2021.01.001
— · multi-finger pose from sEMG alone · prosthetics / neural engineering
21
Zanghieri … Benini · Bologna/ETH · IEEE COINS 2021 · doi:10.1109/COINS51742.2021.9524188
NinaPro DB8 · multi-finger pose from sEMG alone · prosthetics / neural engineering
22
Estimation of joint kinematics and fingertip forces using motoneuron firing activities (index+middle)
Xu, Zheng, Hu · IEEE NER 2021 · par.nsf.gov/servlets/purl/10220198
forearm HD 128ch · borderline (≤3 DoF, sensor fusion, demo or hardware) · prosthetics / neural engineering
23
sEMG-based hand movement regression by prediction of joint angles with RNNs
Koch … Mertins · IEEE EMBC 2021 · DFKI 12721
NinaPro · multi-finger pose from sEMG alone · prosthetics / neural engineering
24
Deep learning with CNN for proportional control of finger movements from sEMG
(EPFL) · IEEE NER 2021 · infoscience.epfl.ch/record/288062
— · multi-finger pose from sEMG alone · prosthetics / neural engineering
25
Liu, Zhang, Gowda · Penn State · WWW 2021 · doi:10.1145/3442381.3449890
forearm (Myo 8ch) · multi-finger pose from sEMG alone · wristband / AR-VR / robotics
26
Liu, Lin, Li · HKUST/… · IMWUT 2021 · doi:10.1145/3478112
forearm (Myo/gForce) · multi-finger pose from sEMG alone · wristband / AR-VR / robotics
27
Finger joint angle estimation based on sEMG signals and deep learning method
— · IEEE conf. 2021 · ieeexplore 9517643
— · multi-finger pose from sEMG alone · prosthetics / neural engineering · title and venue only
2022 (11)
+
28
Sîmpetru … Del Vecchio · FAU · IEEE EMBC 2022 · doi:10.1109/EMBC48229.2022.9870937
forearm+wrist HD 320ch · multi-finger pose from sEMG alone · prosthetics / neural engineering
29
Sîmpetru … Del Vecchio · FAU · bioRxiv 2022 / IEEE TBME 2024 · doi:10.1101/2022.07.29.502064
forearm+wrist HD 320ch · multi-finger pose from sEMG alone · prosthetics / neural engineering
30
Lee, Kim, Park · SNU · IEEE TNSRE 2022 · doi:10.1109/TNSRE.2022.3188275
forearm array · multi-finger pose from sEMG alone · prosthetics / neural engineering
31
Geng … Li · IEEE RA-L 2022 · doi:10.1109/LRA.2022.3169448
NinaPro-type · multi-finger pose from sEMG alone · prosthetics / neural engineering
32
Wang, Tao, Su, Zhang · IEEE Access 2022 · doi:10.1109/ACCESS.2022.3212046
forearm 8ch · multi-finger pose from sEMG alone · prosthetics / neural engineering
33
Comparing efficiency of ANNs in sEMG-based simultaneous and continuous estimation of hand kinematics
Batayneh, Abdulhay, Alothman · Digit Commun Netw 2022 · doi:10.1016/j.dcan.2021.08.002
forearm · multi-finger pose from sEMG alone · prosthetics / neural engineering
34
He, Qin, Koike · Appl Sci 2022 · doi:10.3390/app12083772
32ch sleeve · borderline (≤3 DoF, sensor fusion, demo or hardware) · prosthetics / neural engineering
35
Roy, Kamper, Hu · IEEE ICHMS 2022 · doi:10.1109/ICHMS56717.2022.9980636
HD-EMG · borderline (≤3 DoF, sensor fusion, demo or hardware) · prosthetics / neural engineering
36
— · ICIRA 2022 · doi:10.1007/978-3-031-13841-6_32
— · multi-finger pose from sEMG alone · prosthetics / neural engineering · title and venue only
37
Estimating finger joint angles on sEMG using manifold learning and LSTM with attention
Avian et al. · UGM · Biomed Signal Process Control 2022 · S1746809421006960
NinaPro (Myo) · multi-finger pose from sEMG alone · prosthetics / neural engineering · title and venue only
38
Hinson, Saul, Kamper, Huang · J Biomech 2022 · doi:10.1016/j.jbiomech.2022.111200
4ch · borderline (≤3 DoF, sensor fusion, demo or hardware) · prosthetics / neural engineering
2023 (16)
+
39
Sîmpetru, März, Del Vecchio · FAU · IEEE TNSRE 2023 · doi:10.1109/TNSRE.2023.3295060
forearm HD 320ch · multi-finger pose from sEMG alone · prosthetics / neural engineering
40
Lin … Su · Dalian Maritime/Imperial · IEEE TNSRE 2023 · doi:10.1109/TNSRE.2022.3216528
NinaPro DB2 · multi-finger pose from sEMG alone · prosthetics / neural engineering
41
Roy, Zheng, Kamper, Hu · IEEE TBME 2023 · doi:10.1109/TBME.2022.3232067
HD 2×128 · borderline (≤3 DoF, sensor fusion, demo or hardware) · prosthetics / neural engineering
42
Roy, Kamper, Hu · IEEE Access 2023 · doi:10.1109/ACCESS.2023.3246950
HD-EMG · borderline (≤3 DoF, sensor fusion, demo or hardware) · prosthetics / neural engineering
43
Fan, Vargas, Kamper, Hu · Comput Biol Med 2023 · doi:10.1016/j.compbiomed.2023.107139
HD 128ch · multi-finger pose from sEMG alone · prosthetics / neural engineering
44
Li, Yue, Pan · IEEE TNSRE 2023 · doi:10.1109/TNSRE.2023.3323347
HD-EMG · borderline (≤3 DoF, sensor fusion, demo or hardware) · prosthetics / neural engineering
45
— · IEEE JBHI 2023 · doi:10.1109/JBHI.2023.3234989
— · multi-finger pose from sEMG alone · prosthetics / neural engineering · title and venue only
46
Serdana, Muceli, Farina · Chalmers/Imperial · IJASEIT 2023 · doi:10.18517/ijaseit.13.3.18380
HD 320ch · multi-finger pose from sEMG alone · prosthetics / neural engineering
47
Vangi … Ridolfi · Florence · Machines 2023 · doi:10.3390/machines11070747
forearm (Myo) · multi-finger pose from sEMG alone · prosthetics / neural engineering
48
Wang, Khundrakpam, Vaughan · J Med Biol Eng 2023 · doi:10.1007/s40846-023-00823-x
8ch armband · multi-finger pose from sEMG alone · prosthetics / neural engineering
49
Zanghieri, Benatti, Benini, Donati · IEEE IWASI 2023 · doi:10.1109/IWASI58316.2023.10164372
NinaPro DB8 · multi-finger pose from sEMG alone · prosthetics / neural engineering
50
Leroux, Finkbeiner, Neftci · Jülich · arXiv 2023 · arXiv:2303.11860
NinaPro DB8 · multi-finger pose from sEMG alone · wristband / AR-VR / robotics
51
Liu, Zhang, Gowda · Penn State · IEEE IoT J 2023 · doi:10.1109/JIOT.2022.3214600
forearm (Myo 8ch) · multi-finger pose from sEMG alone · wristband / AR-VR / robotics
52
Graf, Barthet · QMUL · arXiv 2023 · arXiv:2307.10203
armband + camera · borderline (≤3 DoF, sensor fusion, demo or hardware) · wristband / AR-VR / robotics
53
— · ACM conf. 2023 · doi:10.1145/3584376.3584537
— · multi-finger pose from sEMG alone · prosthetics / neural engineering · title and venue only
54
Continuous finger joint angle estimation with a hybrid sEMG-FMG transformer (fusion)
— · Biomed Signal Process Control 2023 · S1746809423004639
sEMG + FMG · borderline (≤3 DoF, sensor fusion, demo or hardware) · prosthetics / neural engineering · title and venue only
2024 (21)
+
55
Salter … Marshall · Meta Reality Labs · NeurIPS 2024 D&B · arXiv:2412.02725
wrist 16ch (2 kHz) · multi-finger pose from sEMG alone · wristband / AR-VR / robotics · builds on emg2pose
56
Sîmpetru, Cnejevici, Farina, Del Vecchio · J Neural Eng 2024 · doi:10.1088/1741-2552/ad3498
forearm HD 320ch · multi-finger pose from sEMG alone · prosthetics / neural engineering
57
Rahimi … Del Vecchio · Tabriz/FAU · arXiv Oct 2024 / IEEE Access 2026 · arXiv:2410.23986
forearm HD · multi-finger pose from sEMG alone · prosthetics / neural engineering
58
Tacca … Friedenberg · Battelle · Sci Rep 2024 · doi:10.1038/s41598-024-64458-x
forearm sleeve 128–150 el. · multi-finger pose from sEMG alone · prosthetics / neural engineering
59
Rolandino … FitzGerald · Oxford/Turin · IEEE TBME 2024 · doi:10.1109/TBME.2023.3346192
forearm HD 96ch · multi-finger pose from sEMG alone · prosthetics / neural engineering
60
Rolandino … FitzGerald · IEEE TNSRE 2024 · doi:10.1109/TNSRE.2024.3490796
dry HD array · borderline (≤3 DoF, sensor fusion, demo or hardware) · prosthetics / neural engineering
61
Cao, Liu, Han, Li · IEEE Trans Mobile Comput 2024 · doi:10.1109/TMC.2024.3439018
wrist-worn EMG · multi-finger pose from sEMG alone · wristband / AR-VR / robotics
62
Lin, Zhang · Front Neurorobot 2024 · doi:10.3389/fnbot.2024.1305605
NinaPro DB2 · multi-finger pose from sEMG alone · prosthetics / neural engineering
63
Lin, He · Front Neurosci 2024 · doi:10.3389/fnins.2024.1306050
NinaPro DB2 · multi-finger pose from sEMG alone · prosthetics / neural engineering
64
Lin, Xiao, Zhao · J NeuroEng Rehabil 2024 · doi:10.1186/s12984-024-01525-4
NinaPro DB2 · multi-finger pose from sEMG alone · prosthetics / neural engineering
65
Chen, Guo, Lin, Jiang, Su · IEEE TNSRE 2024 · doi:10.1109/TNSRE.2024.3400535
NinaPro · multi-finger pose from sEMG alone · prosthetics / neural engineering
66
Zhang … Li · SIAT · Cyborg Bionic Syst 2024 · doi:10.34133/cbsystems.0094
public DB · multi-finger pose from sEMG alone · prosthetics / neural engineering
67
Wang, Tao, Zhang · Sensors 2024 · doi:10.3390/s24020660
forearm 8ch Noraxon · multi-finger pose from sEMG alone · prosthetics / neural engineering
68
Putro, Avian, Prakosa, Mahali · UGM · Biomed Signal Process Control 2024 · doi:10.1016/j.bspc.2023.105447
NinaPro DB5 · multi-finger pose from sEMG alone · prosthetics / neural engineering
69
An … Wang · IEEE Sensors J 2024 · doi:10.1109/JSEN.2024.3423795
— · multi-finger pose from sEMG alone · prosthetics / neural engineering · title and venue only
70
Continuous prediction of hand joint angles based on sEMG and Tf-LSTM
— · IEEE conf. 2024 · ieeexplore 10776268
— · multi-finger pose from sEMG alone · prosthetics / neural engineering · title and venue only
71
An LSTM-Transformer based method for continuous hand pose estimation using sEMG
Huang, Tang · SSRN preprint 2024 · SSRN 4962179
— · multi-finger pose from sEMG alone · wristband / AR-VR / robotics · title and venue only
72
— · ACM conf. 2024 · doi:10.1145/3655755.3655776
— · multi-finger pose from sEMG alone · prosthetics / neural engineering · title and venue only
73
Betthauser … Thakor · JHU/APL · arXiv Dec 2024 · arXiv:2412.17991
sEMG (9 subj.) · multi-finger pose from sEMG alone · prosthetics / neural engineering
74
Berman, Lee, Yin, Huang · IEEE TNSRE 2024 · doi:10.1109/TNSRE.2024.3483150
4ch · borderline (≤3 DoF, sensor fusion, demo or hardware) · prosthetics / neural engineering
75
Enhancing the online estimation of finger kinematics from sEMG using LSTM with attention
— · Biomed Signal Process Control 2024 · S1746809424000296
— · multi-finger pose from sEMG alone · prosthetics / neural engineering · title and venue only
2025 (23)
+
76
Verma · independent · RSS 2025 · arXiv:2508.08269
emg2pose wrist 16ch · multi-finger pose from sEMG alone · wristband / AR-VR / robotics · builds on emg2pose
77
VQ-MyoPose: movement tokenization improves decoding of hand kinematics from sEMG wristbands
Lovecchio … Verstraten · TU Delft/Imperial · ICCV 2025 HANDS workshop · hands-workshop.org VQ-MyoPose
emg2pose wrist 16ch · multi-finger pose from sEMG alone · wristband / AR-VR / robotics · builds on emg2pose
78
Xiao … Gao · UIST 2025 · arXiv:2510.04122
watch 1ch EMG + ring IMU · borderline (≤3 DoF, sensor fusion, demo or hardware) · wristband / AR-VR / robotics
79
Kovalev … Timčenko · ALVI Labs · arXiv 2025 · arXiv:2502.21256
forearm (Myo 8ch) · multi-finger pose from sEMG alone · wristband / AR-VR / robotics
80
Spacone … Cossettini · ETH/Bologna · arXiv 2025 · arXiv:2510.02000
8ch armband + A-mode US · borderline (≤3 DoF, sensor fusion, demo or hardware) · wristband / AR-VR / robotics
81
Lin … Guo · IEEE TNSRE 2025 · doi:10.1109/TNSRE.2024.3514938
NinaPro DB2 · multi-finger pose from sEMG alone · prosthetics / neural engineering
82
Lin, Zhang, Zhao · Sci Rep 2025 · doi:10.1038/s41598-025-16268-y
NinaPro + 192ch HD online · multi-finger pose from sEMG alone · prosthetics / neural engineering
83
Lin, Zhou · Appl Sci 2025 · doi:10.3390/app152011078
NinaPro DB2 · multi-finger pose from sEMG alone · prosthetics / neural engineering
84
Guo … Sheng · SJTU · Sci Data 2025 · doi:10.1038/s41597-025-04749-8
forearm+wrist HD 448ch · multi-finger pose from sEMG alone · prosthetics / neural engineering
85
Xiao et al. · CHI 2025 EA · doi:10.1145/3706599.3720220
wrist EMG + ring IMU · borderline (≤3 DoF, sensor fusion, demo or hardware) · wristband / AR-VR / robotics · title and venue only
86
Luxembourg … Hanein · Tel Aviv Univ. · APL Bioeng 2025 · doi:10.1063/5.0270645
soft printed 16-el. forearm array · multi-finger pose from sEMG alone · prosthetics / neural engineering
87
Finger joint angle and gesture estimation under dynamic hand position with a soft printed electrode array
Luxembourg … Hanein · EMBC 2025 (likely) · PubMed 41336703
soft printed array · multi-finger pose from sEMG alone · prosthetics / neural engineering · title and venue only
88
Chen et al. · UEC · Biomed Signal Process Control 2025 · doi:10.1016/j.bspc.2025.107833
— · multi-finger pose from sEMG alone · prosthetics / neural engineering
89
Deep LSTM networks for kinematic estimation of finger interphalangeal joint angles from sEMG
(Amirkabir Univ.) · Scientia Iranica 2025 · article_23808
NinaPro DB2 · multi-finger pose from sEMG alone · prosthetics / neural engineering · title and venue only
90
Gilstrap … Lee · IEEE Access 2025 · doi:10.1109/ACCESS.2025.3528902
sEMG · borderline (≤3 DoF, sensor fusion, demo or hardware) · prosthetics / neural engineering
91
Pan, Yan, Yue, Li · Med Biol Eng Comput 2025 · doi:10.1007/s11517-025-03321-1
HD-EMG neural drive · borderline (≤3 DoF, sensor fusion, demo or hardware) · prosthetics / neural engineering
92
Simultaneous and proportional control based on an enhanced musculoskeletal model (3 DoF wrist + MCP)
Pan, Liu, Wang, Li · IEEE TNSRE 2025 · doi:10.1109/TNSRE.2025.3543912
sEMG + NMF · borderline (≤3 DoF, sensor fusion, demo or hardware) · prosthetics / neural engineering
93
Pan, Ding, Zhao, Mu, Li · Med Biol Eng Comput 2025 · doi:10.1007/s11517-024-03207-8
4 muscles · borderline (≤3 DoF, sensor fusion, demo or hardware) · prosthetics / neural engineering
94
Andreas … Beckerle · FAU · Sci Data 2025 · doi:10.1038/s41597-025-05852-6
12ch Delsys + IMU · multi-finger pose from sEMG alone · prosthetics / neural engineering
95
Finger joint angle from single-channel sEMG via improved BPNN (collective four-finger angle)
Zhang, Wang, Zhao · IAENG IJCS 2025 · IJCS_52_4_17
single channel · borderline (≤3 DoF, sensor fusion, demo or hardware) · prosthetics / neural engineering
96
Molinari, Elias · UNICAMP · arXiv Dec 2025 · arXiv:2512.13870
forearm HD 128ch · multi-finger pose from sEMG alone · prosthetics / neural engineering
97
Baracat, Manneschi, Donati · IEEE NICE 2025 · doi:10.1109/NICE65350.2025.11065840
NinaPro DB8 · multi-finger pose from sEMG alone · wristband / AR-VR / robotics
98
Modality-wise multimodal transfer learning: dual-stream CNN-Transformer for ACC-sEMG finger joint angles
— · Biomed Signal Process Control 2025 · S1746809425009565
sEMG + ACC · borderline (≤3 DoF, sensor fusion, demo or hardware) · prosthetics / neural engineering · title and venue only
2026 (to 12 Oct) (18)
+
99
Hadidi … Kao · UCLA · arXiv Mar 2026 · arXiv:2603.08212
emg2pose wrist 16ch · multi-finger pose from sEMG alone · wristband / AR-VR / robotics · builds on emg2pose
100
Zhao, Li, Wang, Zhang · Sharpa/SJTU · arXiv Mar 2026 · arXiv:2603.05861
forearm 8ch gForce · multi-finger pose from sEMG alone · wristband / AR-VR / robotics
101
Colot … Bontempi · ULB · arXiv Apr 2026 · arXiv:2604.22499
forearm 8ch armband (+ emg2pose eval) · multi-finger pose from sEMG alone · wristband / AR-VR / robotics · builds on emg2pose
102
Xi … Zhou · Tsinghua · arXiv May 2026 · arXiv:2605.05712
wrist 2×8ch (2 kHz) · multi-finger pose from sEMG alone · wristband / AR-VR / robotics
103
Xie, Cheung · arXiv May 2026 · arXiv:2605.30127
emg2pose wrist 16ch · multi-finger pose from sEMG alone · wristband / AR-VR / robotics · builds on emg2pose
104
Li … Shu · arXiv Sep 2026 · arXiv:2609.07495
HD sEMG armband + IMUs · borderline (≤3 DoF, sensor fusion, demo or hardware) · wristband / AR-VR / robotics
105
Magni … Barmpas · Imperial · arXiv Oct 2026 · arXiv:2610.05235
emg2pose wrist 16ch · multi-finger pose from sEMG alone · wristband / AR-VR / robotics · builds on emg2pose
106
Ge, Zhang, Li, Yang · arXiv Oct 2026 · arXiv:2610.06970
EgoEMG wrist sEMG + RGB · borderline (≤3 DoF, sensor fusion, demo or hardware) · wristband / AR-VR / robotics
107
Wang … Wu · arXiv Oct 2026 · arXiv:2610.07713
emg2pose wrist 16ch · multi-finger pose from sEMG alone · wristband / AR-VR / robotics · builds on emg2pose
108
Chen, Wang, Ji, Li · Adv Intell Syst 2026 · doi:10.1002/aisy.202500590
sEMG · multi-finger pose from sEMG alone · prosthetics / neural engineering
109
Abdelall … ElHady · Sci Rep 2026 · doi:10.1038/s41598-026-70646-8
EMG + IMU · borderline (≤3 DoF, sensor fusion, demo or hardware) · prosthetics / neural engineering
110
Manneschi, Ellis, Donati · Sheffield · IEEE TNNLS 2026 · doi:10.1109/TNNLS.2026.3662308
NinaPro DB8 · multi-finger pose from sEMG alone · prosthetics / neural engineering
111
Ghosh … Narayan · IEEE CoDIT 2026 · doi:10.1109/CoDIT70676.2026.11631209
NinaPro DB1 · multi-finger pose from sEMG alone · prosthetics / neural engineering
112
Yan … Li · J Electron Inf Technol 2026 · doi:10.11999/JEIT250033
8ch sEMG (DB8/DB5) · multi-finger pose from sEMG alone · prosthetics / neural engineering
113
(Rolandino/FitzGerald group, likely) · J NeuroEng Rehabil 2026 · doi:10.1186/s12984-026-01881-3
HD-sEMG · multi-finger pose from sEMG alone · prosthetics / neural engineering · title and venue only
114
Gilardini, Ma, Nazarpour · Edinburgh · arXiv Jul 2026 (ICML 2026) · arXiv:2607.04820
NinaPro DB8 · multi-finger pose from sEMG alone · wristband / AR-VR / robotics
115
Andreas et al. · Sci Rep 2026 · doi:10.1038/s41598-026-59979-6
sEMG + IMU + FMG · borderline (≤3 DoF, sensor fusion, demo or hardware) · prosthetics / neural engineering · title and venue only
116
Chen Y et al. · IEEE TNSRE 2026 · doi:10.1109/TNSRE.2026.3706311
4ch sEMG + B-mode ultrasound · borderline (≤3 DoF, sensor fusion, demo or hardware) · prosthetics / neural engineering · title and venue only
Method notes
List A was compiled from arXiv, OpenReview, project pages and the Awesome-Touch and Awesome-Force-Tactile-VLA lists; 64 of 68 entries were verified against the arXiv abstract or full text.
List B was compiled from about 140 web searches plus arXiv, DOAJ and publisher pages; bibliographic APIs (OpenAlex, Semantic Scholar, Crossref, dblp, PubMed) were unreachable, so coverage of IEEE conference proceedings and subscription journals is incomplete, especially for 2018–2019 and 2025–2026.
↑