Multimodal Tactile Display Device for Biomedical Applications


"NI provides the necessary tools for any engineering solution. It provides hardware and software that can be easily used to build the system interface and to process the acquired data professionally."

- Nader A. Mansour, Mechatronics and Robotics Department, Egypt-Japan University of Science and Technology

The Challenge:
Developing a tactile display system to show the shape and stiffness of tissues and organs inside the abdomen, where surgeons have no direct access to check them during surgical operations.

The Solution:
Using the NI PXI-6259 high-speed multifunction M Series data acquisition (DAQ) board and NI LabVIEW system design software to develop a control technique that regulates the shape and stiffness of the shape memory alloy springs to represent the shape and stiffness of tissues or organs of interest.

Nader A. Mansour - Mechatronics and Robotics Department, Egypt-Japan University of Science and Technology
Ahmed M. R. Fath El-Bab - Mechatronics and Robotics Department, Egypt-Japan University of Science and Technology - on leave from Mechanical Engineering Department, Assiut University
Mohamed Abdellatif - Mechatronics and Robotics Department, Egypt-Japan University of Science and Technology

Tactile display devices are human computer interfaces that can display the tactile information of real objects, such as shape, softness, surface texture, roughness, vibration, and temperature, in virtual environments. Figure 1 shows the laparoscopic surgery display of an abnormality inside a patient’s abdomen, which is an important application of a tactile display.


Figure 1. Tactile Perception System

Conceptual Design

The tactile display device consists of a 5X5 pin matrix that can be configured to represent the shape of real objects while having a varying stiffness. We built the design concept on a simple use of two shape memory alloy (SMA) springs in series for each display pin of the matrix to control the shape and stiffness of an object. One of these springs, the elongation spring, displays the shape/elongation while the other, the stiffness spring, displays the stiffness of the real object, as shown in Figure 2.

We selected the SMA material as an actuator because it usually has a number of changing variables describing its characteristics,1 including temperature, generated force (stress), displacement (strain), and electrical current. Both the displacement and stress act as output state variables in most applications in which the SMA is used as a part of actuation. The temperature and electrical currents act as internal state variables representing the percentage of SMA phase transformation.

Figure 2. Tactile Display Pin Unit

Figure 3. Tactile Display System Setup

Experimental Setup and System Integration

Figure 3 shows the system setup of the tactile display pin under test with its complete sensory system. For each pin of this array, the system detects displacement and stiffness as output variables and the current as a control variable to change the temperature of the SMA, which is the direct parameter that changes the mechanical properties of elongation and stiffness of the SMA springs.

The tactile display matrix contains a huge amount of feedback and control data going to and from the tactile display system. Consequently, we selected the NI PXI-6259 DAQ device to handle this amount of data. Figure 4 shows the system integration of the sensors used to detect the process variables (elongation and stiffness) and the control variable (current) using the NI PXI-6259 DAQ board to have a closed-loop system that is ready for applying the control technique.

Control System

As previously mentioned, each pin consists of two decoupled SMA springs—the elongation spring and the stiffness spring. Changing the elongation spring has no effect on the stiffness spring. Figure 5 shows the closed-loop block diagram of the elongation control system. The NI PXI-6259, the NI SCB-68 connector block, LabVIEW, and the LabVIEW Control Design and Simulation Module significantly helped our control development technique. The built-in proportional integral derivative (PID) autotuning tool played a large role in the controller gains selection. Additionally, the data acquisition software calibrated the signals out of the sensors to match the range of displacement and stiffness of the process variables.

We applied the control system using LabVIEW with a sampling time of 50 ms, which was more than enough for the slow temperature systems. We let the power supply deliver a current of up to 2 A, which was needed to speed up the system, especially when there were a lot of errors and significant delays as a result of the slow response of the temperature systems.

Figure 5. Block Diagram of the SMA Elongation Spring Control System

Figure 6. Tracking Response of the PI Controller

We used the Arduino kit and the SMA driving circuits to amplify the control output to suit the high current required to drive the SMA springs. The tactile display system showed good response and used the PI controller in the existence of the cooling fan to solve the tracking problem of the position control. It could display displacement of 5 mm in 5 seconds. We could use the same control technique to characterize and control the displayed stiffness range of interest.

Results and Benefits

The developed system, a novel solution to reproduce both the stiffness and shapes of real objects simultaneously, is a promising solution for virtual reality environments and may have many applications in surgical operations or mobile applications. We found it easier to use NI hardware and software than any other microcontroller because the NI systems offered better results and more repeatable responses for the same inputs. Real-time operation was also a key benefit. Lastly, by using a complete NI system, all of the components were highly compatible with each other and data  transferred smoothly.

Future Work

In the future, the same concept applied on this tactile display pin can be implemented on the matrix level with the NI DAQ card that has 16 differential or 32 single-ended analog input channels and 4 analog output channels with a sampling rate of 1 MS/s, which is more than enough for such a slow temperature system.


  1. 1]  K. Ikuta, “Micro/miniature shape memory alloy actuator,” in Proceedings, IEEE International Conference on Robotics and Automation, 1990, pp. 2156-2161.

This work was funded by Mitsubishi International corporation, which is gratefully acknowledged.

Author Information:
NaderA. Mansour
Mechatronics and Robotics Department, Egypt-Japan University of Science and Technology
New Borg El-Arab, Alexandria
Tel: +2 0100 990 5122

Bookmark and Share

Explore the NI Developer Community

Discover and collaborate on the latest example code and tutorials with a worldwide community of engineers and scientists.

‌Check‌ out‌ the‌ NI‌ Community

Who is National Instruments?

National Instruments provides a graphical system design platform for test, control, and embedded design applications that is transforming the way engineers and scientists design, prototype, and deploy systems.

‌Learn‌ more‌ about‌ NI