About/关于
人类社会的周围环境具有高速变化的动态特性，若对此变化的环境对象进行实时观测与交互，传统的采集方法和检测手段将面临很大的瓶颈。
动态图像感知（Dynamic Image Perception）名字的由来，源自日本東京大学石川正俊教授（Masatoshi Ishikawa）研究室，时任组长奥宽雅博士（Hiromasa Oku）发起的Dynamic Image Control课题组名，研究室倡导通过光、机、电、信息一体化的全局调控方式，在时间维度和空间维度上超越人类视觉感知。
Object/着眼点
研究方向围绕超高速视觉展开，以高速对焦和高速追踪为着眼点，解决以高速移动物体为采集对象时的“看不见”和“看不清”的“卡脖子”难题， 并在此基础上进一步拓展高速高清机器视觉和高速动态显示交互技术。
Originality/独创性
凝练并融合了光、机、电、信息工程领域的科学知识，以周边高速变化的动态信息为观测对象，发展新一代视觉信息感知、分析处理和显示技术。
Research and Applications/研究范围及应用场景
- 光学仪器、自适应光学设备（变焦/调焦液体透镜、智能老花眼眼镜；超高速视觉追踪）
- 先进制造、智能制造（高器视觉、机器人导航控制；柔性传感器、精密器件抓取）
- 安防、监控摄像设备（高速目标追踪、可疑目标提示）
- 新型显示、虚拟/增强/混合现实（动态光雕投影显示技术；三维车载抬头显示器；医疗手术辅助导航）
In Detail/具体展开
1 Active Vision/主动视觉
1-1 Variable Focus Lens/可变焦液体透镜
In order to change focus with traditional solid lenses, which have fixed optical properties, two or more lenses have to be jointly moved mechanically. In contrast, a variable focus lens can dynamically control its focal length by only using a single lens element. Liquid-filled variable focus lenses are based on the physical deformation of refractive surfaces, which changes their curvature.
We proposed a novel variable focus lens with a large optical aperture. The lens consists of two chambers separated by a membrane. The chambers were infused with two different liquids characterized by their similar density but different refractive indices. Thus its deformation was in the interface between the two liquids, and it acted as a refractive surface due to the difference in refractive index of these liquids. If one fluid was made to flow into and out of its chamber, while the other was locked, the lens could shift its power dynamically by means of a syringe pump.
|
|
自研可变焦距液体透镜（大口径版） A photograph of the lens prototype. | 可变焦距液体透镜的性能测试展示 Experimental view of its variable focus performance. |
|
|
可便携式可变焦距液体透镜 Waearable liquid-based tunable lens | 可便携式液体透镜（东京电视台专题报道） Waearable liquid-based tunable lens |
Reference/ 参考
Zenghong Duan, Lihui Wang*, Zhi Li, Jian Fu, Susheng Fu, Boqian Chen, Yuxun Chen, and Yong Zhao, “Dynamic performance of a membrane-based variable focus lens with a large aperture,” Appl. Opt. 62, 4609-4617 (2023)[DOI:10.1364/AO.486278]
Lihui Wang, Hiromasa Oku, Masatoshi Ishikawa, Paraxial ray solution for liquid-filled variable focus lenses, Japanese Journal of Applied Physics, Volume 56, Number 12, 122501 (2017)[DOI:10.7567/JJAP.56.122501]
Lihui Wang, Hiromasa Oku, Masatoshi Ishikawa, An improved low-optical-power variable focus lens with a large aperture, Optics Express, Vol.22, Issue 16, pp. 19448-19456 (2014)[DOI:10.1364/OE.22.019448]
Lihui Wang, Hiromasa Oku, Masatoshi Ishikawa, Variable-focus lens with 30 mm optical aperture based on liquid-membranes-liquid structure, Applied Physics Letters, Vol.102, 131111 (2013)[DOI:10.1063/1.4800603]
1-2 High-speed Active Visual Tracking/高速主动视觉追踪
在追踪具有高度非线性的高速随机运动目标时，传统光电主动视觉系统存在目标丢失、视觉追踪失效的问题。课题组通过搭建一套毫秒级光电主动视觉系统。采用光电云台结构、感兴趣区域提取、并行计算、高速运动目标追踪算法，实现毫秒级目标跟踪。
|
|
| 高速视觉追踪效果图. （a）目标从右至左运动过程; (b) 目标下落-碰撞地面-弹起的高动态过程。 | 球类高速运动的实时追踪，高速主动视觉系统，可以时刻保持观测对象在计算机视野中心（ball real-time tracking） |
|
|
| 6自由度实时追踪测试（ test for 6-dof tracking） | 乒乓球拍6自由度实时追踪（tracking pingpang pat 6-DOF） |
Reference/参考
Jiaqi Li, Lin Li, Lihui Wang*, Lei LI, Shaoyong Li, and Masatoshi Ishikawa, Adaptive milliseconds tracking and zooming optics based on a high-speed gaze controller and liquid lenses, Optics Express, Vol.32, Issue 2, pp. 2257-2270 (2024) [DOI:10.1364/OE.512003]
Ruimin Cao, Jian Fu, Hui Yang, Lihui Wang*, and Masatoshi Ishikawa, “Robust optical axis control of monocular active gazing based on pan-tilt mirrors for high dynamic targets,” Optics Express 29, 40214-40230 (2021) [DOI: 10.1364/OE.439083]
3 High-Speed 3D & Depth Imaging/成像
A large open aperture in an optical system can capture high-resolution images but yields a shallow depth of field. In order to keep the high-resolution and enlarge the DOF at the same time, back-and-forth movement of the lens should be driven by the rack and pinion motion of a motor. However, continual forward and reverse rotation is a high-power-consumption task, because positive and negative current are used alternately to control the motor, which quickly triggers a safety stop to prevent overheating. Moreover, it is quite difficult to achieve high-speed responses in such conditions.
|
|
| 倾斜放置的芯片（传统成像方式导致局部信息清晰，局部星系模糊的典型案例） | 高速焦点扫描成像方式，短时间内采集12张不同景深图像（左下），通过自研图像处理算法，实时生成全焦点图（右上） |
|
|
| 眼底模型（大景深物体） | 实时对焦点扩展景深成像（shape from focus） |
Reference/参考
Huayu Cheng, Lihui Wang*, Satoshi Tabata, Yuan He, Yan Hu, Jiang Liu and Zhiwei Mou, High-speed all-in-focus 3D imaging method based on the liquid lens focus scanning, Applied Optics, Vol.63, Issue 21, pp. 5602-5610 (2024) [DOI:10.1364/AO.523864]
Lihui Wang, Jianjiang Cui, Satoshi Tabata, Masatoshi Ishikawa, Low-cost, readily available 3D microscopy imaging system with variable focus spinner, Optics Express, Vol.26, Issue 23, pp. 30576-30587 (2018).[DOI:10.1364/OE.26.030576]
4 High-Speed 3D Projection Mapping/显示
Projection mapping (PM) is attractive as a fundamental technology for the advancement of various subjects, such as media art, entertainment, and augmented reality. However, conventional projectors have a shallow depth of field (DOF); therefore, sharp images are only visible in the limited depth range. In the case of dynamic projection mapping (DPM), which can project images on the surface of the moving objects, the shallow DOF limits the permissible motion of the object, because the projected images become blurred when the object is outside the DOF.
Our laboratories have developed a high-speed focal tracking projection system, which includes the technologies of high-speed vision, high-speed projector, and high-speed variable focus optics. In this system, the variation of the object’s distance and posture was captured using the high-speed vision technology that served as immediate feedback to the liquid lens and high-speed projector. As a result, the focal distance is compensated, and the projected images are updated in real-time to fit the moving object. Therefore, a well-focused image projection was achieved even when the motion involved large depth range movement.
This system could ensure that the projected images were sharp and clear at variable distances, while the object was moving dynamically in a large three-dimensional area. Hence, this approach can be effectively applied to applications such as Volume Slicing Display. Furthermore, it can turn any physical surface into an interactive display, and enable the manipulation of their appearance to provide detailed information. Our system provides the essential technology for expanding such applications.
|
|
| Prototype of projection tracking system using a liquid lens. (Ver.7) (2018-) | Prototype of projection tracking system using a liquid lens. (Ver.2) (2019-) |
|
| Volume Slicing Display: change images based on the distance. |
|
|
|
提案系统能实时保持投影对焦清晰（左侧图为本系统）our projector is always in focus (left) | 即使投影对象时刻旋转，投影信息能根据旋转反馈调整/Target rotationcan be detected self adjust projection | 医疗用三维数据信息能够在三维空间内做高清切片投影展示/ 3D CT data can be projected and shown with slicing display |
本技术与2019年11月19日获得日本经济产业省数字信息科技创新奖、美国计算机图形学会特等奖、日本CGWorld赞助商奖三项奖励。此次授奖也是日本数字信息展，首次将三个奖项同时授予同一项技术。
The following three awards were given to the project “High-speed focal tracking projection system based on liquid lens”.
ACM SIGGRAPH Special Prize, (Association for Computing Machinery)
Innovative Technologies 2019, (Digital Content Association of Japan)
Innovative Technologies 2019, Sponsor Award (CGWORLD) 2019, (Digital Content Association of Japan)
Reference/ 参考
Lihui Wang*, Satoshi Tabata, Hongjin Xu, Yunpu Hu, Yoshihiro Watanabe, and Masatoshi Ishikawa, Dynamic depth-of-field projection mapping method based on a variable focus lens and visual feedback, Optics Express, 31(3), pp. 3945-3953 (2023) [DOI:10.1364/OE.478416]
Lihui Wang, Hongjin Xu, Satoshi Tabata, Yunpu Hu, Yoshihiro Watanabe, and Masatoshi Ishikawa: High-Speed Focal Tracking Projection Based on Liquid Lens, ACM SIGGRAPH 2020 Emerging Technologies (SIGGRAPH ’20) (Virtual Event, USA, 2020.8.24-28) [DOI: 10.1145/3388534.3408333]
Lihui Wang, Hongjin Xu, Yunpu Hu, Satoshi Tabata, Masatoshi Ishikawa, Dynamic Depth-of-Field Projection for 3D Projection Mapping, ACM CHI Conference on Human Factors in Computing Systems (CHI’19) (Glasgow, Scotland, UK. 2019.05.05-09) [DOI: 10.1145/3290607.3313246]
5 3D Augmented Reality Head-Up-Display / 三维增强现实车载抬头显示器
Head-Up-Display (HUD) enables a driver to view information with his head positioned “up” and looking forward, instead of angled down looking at lower instruments. By adding the Augmented Reality technology, targets, like people and cars, can be marked to alarm to the drivers to avoid the potential accidents.
Traditional 2D AR HUD projects information messages at a certain distance away from the driver. It asks a driver to observe the projection along the optical axis at a certain point. When the driver moves his head, a miss-matching projection occurs between the projected data and the target in the real world.
In this 3D AR HUD technology, a 3D virtual display can be projected in front of the driver. AR messages will be dynamically projected according to the 3D locations of the targets. In our 3D HUD, a virtual display is projected into a three-dimensional world, so there will be no mismatch when the driver moves.
The following demos were recorded by two cameras, which were placed at different places. When the camera was placed along the optical axis, 2D and 3D markers were all perfectly matched. When the camera was placed at an angle to the optical axis, a mismatch was found in 2D HUD, but 3D HUD was still well matched.
This work was conducted by a collaborate research project with Ishikawa Laboratory and Konica Minolta Inc..
|
|
|
| A photo of 2D HUD projection. All the 2D HUD markers were projected at a certain distance. | A photo of 3D HUD projection. 3D HUD markers were projected at different distance. This photo were token when it focused at close. | A photo of 3D HUD projection. 3D HUD markers were projected at different distance. This photo were token when it focused at far. |
6 Smart Actuator and sensor/软体驱动器和传感器
力触觉感知和交互作为人-机交互的主要方式之一，其通常需要人工设备具备良好的柔性触觉感知能力。为丰富人工设备所能感知的触觉信息，基于混合感知机理的触觉传感器近年来受到了广泛关注。然而，对于可穿戴或柔性人工设备而言，开发具有简单结构、易制备、成本低、功耗低、易于维护和集成的触觉传感器仍然是一项艰巨挑战。
|
|
| 自研基于液体透镜的柔性触觉感知器，抓取柔性物体 | 基于力反馈控制实现对超软物体的无损抓取 |
Reference/参考
Hui Yang, Tianzhao Bu*, Wenbo Liu, Jiaqi Liu, Yunzhi Ling, Meixia Wu, Weirui Liu, Changan Wang*, Xifeng Gao*, Lihui Wang*, A novel triboelectric-optical hybrid tactile sensor for human-machine tactile interaction, Nano Energy, 125, pp. 109592 (2024) [DOI:10.1016/j.nanoen.2024.109592] (IF=17.6)
Ronghua Hu, Hui Yang, Lihui Wang*, and Longfei Fan, Flexible optical tactile sensor based on a liquid-membrane lens structure, Applied Optics, 62(26), 6952-6960 (2023) [DOI:10.1364/AO.496741](IF=1.905)
Hui Yang, Jiaqi Liu, Wenbo Liu, Weirui Liu, Zilong Deng, Yunzhi Ling, Changan Wang, Meixia Wu, Lihui Wang*, and Li Wen*, Compliant Grasping Control for a Tactile Self-Sensing Soft Gripper, Soft Robotics, (2023) [DOI:10.1089/soro.2022.0221] (IF=7.784)
Media and others （媒体报道等）
1. Dynamic Projection Mapping（动态光雕投影）
1) ITmedia News. 液体レンズを用いた高速焦点追従投影システム[https://dip-labs.com/news/20210128-01/] (2021.01) (日本語)
译：ITmedia News，基于液体的动态光雕投影系统。
2) 中国科技网/科技日报. “液体透镜带来光学镜头革命” [http://www.stdaily.com/index/kejixinwen/2019-11/21/content_817911.shtml?from=singlemessage](2019.11) (in Chinese)
3) 日刊工業新聞. “動く物体に絵を投影できるプロジェクション技術、どんな用途に使う”[https://newswitch.jp/p/20672?fbclid=IwAR3MP-srPICwf7SsFN333tQQ-9pA-Jddmu1eq8ElD61jKqIMw6nge9yeGrU] (2020.01) (in Japanese)
译：日刊工业新闻。可在移动物体表面绘画的光雕投影技术，潜在用途是？
2. Variable Focus Lens（可变焦液体镜头）
1) テレビ東京ワールドビジネスサテライトトレンドたまご [オートフォーカスの老眼鏡!?] [https://txbiz.tv-tokyo.co.jp/wbs/trend_tamago/post_124134/] (2017.01) (in Japanese)
译：东京电视台，全球财经卫视Trend Tamago专题，自动对焦老花眼眼镜！
3. 3D AR Head-Up-Display for ADAS Automotive（车载三维抬头显示器）
1) Konica Minolta Release. “Konica Minolta Develops the World’s First* Automotive 3D Augmented Reality Head-up Display” [https://www.konicaminolta.com/newsroom/2017/0227_01_01.html]. (2017.02)
译：柯尼卡美能达，世界首发车载三维增强现实抬头显示器
2) コニカミノルタニュースリリース.“コニカミノルタ 世界初*の車載用3D 拡張現実ヘッドアップディスプレイを開発” [https://www.konicaminolta.com/jp-ja/newsroom/2017/0113_01_01.html]. (2017.01) (in Japanese)
译：柯尼卡美能达，世界首发车载三维增强现实抬头显示器
3) 日本経済新聞. “車載用3D拡張現実ヘッドアップディスプレーを開発” [https://www.nikkei.com/article/DGXLRSP433224_T10C17A1000000/].(2017.01) (in Japanese)
译：日本经济新闻，车载三维增强现实抬头显示器
更加详细的研究信息可参考：diplabs.wordpress.com