Development of an Android Recognition Open-Loop Tracking Control of Writing Robotic Arm

: Writing is one of the medium of the manifestations of the expression which is staged by going through many evolutionary phases for the time being. And most importantly robots and technology are dramatically meliorated in the 21st century. So, robotic writing hand was introduced with the collaboration with human brain and technology. The whole methodology is staged for the development of a low-cost robotic arm able to draw in an arbitrary 2-D plane through an android application. The Android app will take drawing inputs on its Android screen and the data bits are transferred to the robot through wireless standard IEEE 802.15.1 communication technology. The human friendly robotic arm uses the polar coordinate system instead of the Cartesian coordinate system for determining the end-effector position. The modified slider model with the rope-pulley mechanism makes the robot compact. The description of the mechanical and electrical design along with the practical and simulation results are presented in this paper.


Introduction
The idea of robots has come to the role of human assistants. Writing is one of the conclusive statuses of human intelligence that it is not facile to matriculate robots. Despite this, Writing Robot is a favorite apt of researchers and the University, Research Institutes and companies are developing a lot in this stratum.
The Humanoid robots are now employed in services, home works and other human activities (Barakeh et al., 2019). Thus, it is required to furnish the robots with the capability to move in a human-like fashion. As a consequence, the handwriting is regarded as an important human task and accordingly, it draws the attention of researchers working in physiology, biomechanics and related fields (Veljko, 2008). Handwriting is engaged with different level of motion control involving both the dynamics and kinematics. If humanoids continue to elevate the human-likeness, the robotic handwriting is a must adoptable task. The application of this writing robotic system is not bounded only in the writing. Rather it can perform the act of industrial welding, customize painting on a large scale, medical recovery exercise and so on. However, this work is viewed as a testbed for many more 2-D robotic arm applications.
In this study, a writing robotic system is developed which allows helping a user to write in a 2D arbitrary plane with a large scale by writing on the android screen. The Android app is developed in Android studio environment. The user interface (UI) is designed to take input while the user drags finger on the android screen. Since the output drawing has a scaling factor, the Bresenham line drawing algorithm is applied to generate a path between two adjacent input pixels. Bluetooth communication between the Android device and the robotic system is set up by using the HC-05 Bluetooth module. This technology provides a cheap and convenient way of communication.  Figure 1 shows the basic diagram of the proposed system. Using the polar coordinate system, the use of rail in mechanical design is estimated. Instead of this, a prismatic joint guided by a rope-pulley mechanism is adopted. So that, the robot cannot occupy more space and can be folded easily when it is not being used. For controlling the movement of the pen, a rack and pinion mechanism is chosen. And for controlling the eraser position, a generic mechanism of slider-crank linkage is adopted.
At first, the process of the system begins with a custom made Android application. After opening the Android app, an opening page appears. It requests the consent from the user to enter the main screen. In the beginning, the main robotic system is paired with the Android application by clicking the connect button. After pairing, a confirmation dialogue appears on the dialogue box. Now the Android is ready to send data to the system. The system is operated with two modes. One is the drawing mode and the other is erasing mode. If a user selects the drawing mode, the system allows the user to draw. When the user drags his finger on the approved area of the android screen, the instantaneous trajectory information of the finger is sent to the Arduino through Bluetooth technology. This information is a string type data and contains three information: x and y value of the pixel and the mode index number. These values are separated by a separator. The Arduino receives the string and split it. Then it maps the x and y values to the values of the robot's working region. After that, the Arduino calculates the distance and the angle by using the kinematic equations. Due to the mechanical design, the robot cannot reach exactly at the origin. Thus, a number is trimmed from the original distance. The servo angle for both base and radius servo is also calculated afterward. In the drawing mode, the pen touches the drawing plane as long as the user touches the android screen. If the user detached the finger from the screen, the Arduino commands the system to uplift the pen.
The operation of the erasing mode is exactly the same as the writing mode except the distance behind the pen. Thus the trimmed distance is different. The user selects the erase button from the app to activate the erasing mode. In this mode, any part of the drawing can be erased. With the Android application, the user can also save the drawing as a picture on the phone memory. The experimental results of the system are evaluated in the results section.
In the previous literature, experts research on the field of robotic writing. A recent signal tracking-based approach addressed in (Chenguang et al., 2015) uses a MYO armband to measure the electromyography (EMG) signal on the forearm and thus instruct the robot to write. In another literature, a hybrid noninvasive EEG-EMG-BCI system is introduced (Gao et al., 2017). This system records the EEG signal, interpret it and send the multidimensional control commands to operate the robot. The advantage of these methods is rapid data acquisition. Besides the advantages, these methods have the nuisance of requiring to append sensors with the muscles. Moreover, the noise due to the user movement affects the signal and thus lowers the signal to noise ratio which consequently decreases the accuracy level. A robot calligraphy system applies human arm gestures to establish a font database and then uses the database to write words and characters (Chao et al., 2017). This system requires a special sensor, complex mechanical design and lengthy training phases. A PLC-based calligraphy robot is designed in (Man et al., 2010). It requires complex mathematical calculations and costly devices like PLC and HMI. Grana et al. proposes an approach, where the desired word converted to a sequence of target points and the control algorithms make the robot movement according to the specifications (Graña et al., 2017). This approach does not allow the robot to plot something in real-time. Low cost visual guided robotic arm is presented in (4). This eye to hand configuration scheme needs image processing. A speech to write robotic arm is introduced in (Graña et al., 2017). In this system, the pronounced word is written by the robotic system. It provides a great writing alternative for physically handicapped people but the speech signal is not always recognizable in the noisy place.
The nobility of this work is to present an easy and convenient way of real-time large scale drawing without using any additional sensor for human gesture recognition. This system has no training phases. Thus, no additional time and the complicated programming is required. A user can draw anything within its working region. In mechanical design, it provides a stable structure. The experimental results manifested in this paper assures the system has extensive control over its performance. The remainder of the paper is constructed as follows. Section 2 describes the working procedure of the system. Section 3 presents the mechanical design and description. In section 4 the electrical design of the system is manifested. The design and development of the Android application are described in section 5. The flow chart of the robot program is presented in section 6. And section 7 analyses the experimental results of the system.

Kinematic Modeling of the System
The kinematic modelling and the coordinate establishment is the key point to achieve the harmonization between the robotic manipulator and the control device. Thereupon, the Denavit Hartenberg (DH) conception where some homogeneous matrices of individual kinematic joints are used to define the transformation of the coordinate. The DH coordinate frames are exhibited in Fig. 2, which comprises the layout of the manipulator's joints, links and the orientations.
In the Fig. 2, the two links of the manipulator are labelled as l1 and l2 and their lengths are 7 cm and 5 cm sequentially. The arm comprises a rotational joint and a prismatic joint. These two joints work concurrently for writing characters in a two-dimensional workspace and thereby acquire a high performance writing. In the threedimensional workspace, the remaining axis is used for lifting the pen up and holding the pen down as per need. For analyzing the inverse kinematics, the DH parameter table is conferred in Table 1. The parameters can be practiced to illustrate the corresponding position and orientations of the links.
From the Table 1, we obtain the transformation matrices to transform from one frame to another. In the last column of the matrices, the 31 matrix gives the position vector of frame i corresponding frame i-1:  As frame 0, 1 and 2 represent the rotating base, the prismatic joint and the end-effector respectively, the transformation matrix from base to the end-effector can be obtained as follows: The constants l1 and l2 are 7cm and 5 cm respectively. So, the position vector (p , py, pz) = (5,0,7). From the equations of forward kinematics, the equations of inverse kinematics can be derived as follows: These equations allow the user to control the robot joints providing the desired position vector as input. Figure 3 represents the experimental setup of the system. The drawing robot is designed to draw different shapes and characters on a two-dimensional plane. The system contains four fundamental modules. This includes the pen and eraser control mechanism, slider, pulley and robot base. At first, the CAD drawing of the robot is created as in Fig. 4. After analyzing the CAD drawing and the material selection, the real robotic system is developed.

Mechanical Design
The base of the robot is made with solid wood and placed in a fixed position. The base servo is positioned on the base and the rest of the system is mounted on the shaft of the base servo with a metallic servo horn. The slider is linked with the pulley through a string or rope. The string starts from one end of the slider turns the pulley one time and ends on the other end of the slider. For gripping the string, a rubberized layer is coated inside the groove of the pulley. This mechanism transforms the rotary motion of servo to produce linear motion in the slider. Thus, the length of the slider varies with the rotation of radius servo.
The slider can prolong up to 20 cm with a complete 180 rotation of radius servo.  For uplifting and down lifting the pen and the eraser, the rack and pinion gear mechanism is adapted. The eraser is positioned 3 cm behind the pen. The up-down lifting position of pen and eraser can be calibrated with the screw. The workspace of the robot is exhibited in Fig. 5.
The robot cannot plot any point involving the radius below 6 cm and above 26 cm. The blue shaded area in the Fig. 5 denotes the working region of the robot. The conferred mechanism of drawing robot brings an easy and convenient way of drawing characters and shapes.

Electrical Design
The robotic system uses the Arduino board to receive data from the Android, for calculation and for controlling the motions of servos. An HC-05 Bluetooth module is connected with the Arduino. The module receives data Workspace from the Android device and sends it to the Arduino through serial communication. Four servo motors supply the necessary joint values for plotting a point on an arbitrary two-dimensional plane. One servo motor produce angle, another servo is used to generate distance and the rest two servos are used to control the position of pen and eraser. Towerpro MG995 digital metal gear servo is selected for each task. The servos are powered from a 1000 m Ah lithium polymer battery and the Arduino is powered from a power bank through a USB cable. The entire circuit diagram is illustrated in Fig. 6

User Interface of Android Application
The user interface of the application is designed with some buttons and dialogue box. Two screens are needed for this graphical user interface. The first one is for the opening page and another is for the main functional page. The left column in the Fig. 7 presents the opening page and the right column presents the main functional page of the Android application. The starting page consists of two buttons named start and exit. By pressing the start button the main functional screen appeared on the Android screen. And in case of pressing the exit button, the application will close. The main functional page is consisted of a drawing box and the buttons named "erase", "draw", "save drawing" and "Connect". The erase and the draw button selects the erase and the drawing mode in the robotic system. The current X and current Y dialogue boxes show the X and Y values of last touched pixel of the drawing box. The box on the bottom right corner shows the connection status between the Android and the robotic system.

Working Procedure of Android Application
By taping the "start" button on the starting page the main function is called and pressing the "exit" button finish () and System. exit (0) proceed. The main functional page consists of a drawing box and four buttons. The "Erase" button makes the drawing with the white value of the "paint" parameter and "Draw" button makes the drawing with the black value of the "paint" parameter. A variable named "mode" is set to 0 if the erase button is clicked and set to 1 if the draw button is clicked. By dragging into the drawing box on the android screen, the drawing is established. "canvas. draw Line (X1, Y1, X2, Y2, 20, paint)" is used for this purpose. A drawing is mainly a structure array consisting of two integer value denoting the pixel value of X and Y. Then, the Brenham's line drawing algorithm connects all the X and Y and made the drawing visible. When a single dot is plotted on the android screen, the Arduino starts to receive the data from android phone and made it visible to the drawing board.
The Android sends the instantaneous X and Y value of a pixel along with the value of the variable mode. These three values are concatenated and make a single string as "instantaneous X: instantaneous Y: mode". Meanwhile, Arduino receives null byte when the screen is not being touched. Bitmap. create Bitmap (v1. get Drawing Cache ()) is applied for taking the screenshot of the particular screen and bitmap. compress (Bitmap.CompressFormat.JPEG, quality, output Stream) is implemented for saving the image in a compressed format of JPEG if "save drawing" button is touched.

Data Communication between Android and Robotic System
This section depicts the architecture and procedure of communication used in the robotic system is shown in Fig. 8. The IEEE 802.15 is confirmed as the communication medium as the system requires a low energy technology with cable replacement. This low energy communication method consists of four main groups named as Bluetooth core, cable replacement, telephony control protocols and adopted protocols. Meanwhile, the Bluetooth core mainly comprises baseband (BB), Logic Link Control (LLC) and service discovery protocol. The BB layer fills the requirements of BB and LLC makes a physical radio recurrence interface between devices in this manner making a piconet where power level operation is handled by layer where layer supervises power level ventures. Logic link control serves not only the purpose of segmentation but also the reassembly of large data packets. Correspondingly, Service Discovery Protocol (SDP) is a network-level protocol that executes the task of automatic detection of devices and services offered by these devices on a computer network for degrading the configuration troubles from users. Furthermore, L2CAP (logical link control and adaptation protocol) layer, above the Host Controller Interface (HCI) in the upper stack plays a central role in making a communication bridge between baseband customer and baseband supplier of Bluetooth stack. Besides, for handling the communication between the module and the host, HCI command packets are used such as the even packets and data packets. Moreover, a serial interface with Object Exchange (OBEX) and Wireless Application Protocol (WAP) is provided by the radio frequency communication (RFCOMM) which is utilized legitimately by numerous telephony related profiles as a transporter for commands, as well as denoting a transport layer for OBEX over Bluetooth. Additionally, Telephony Control Specification Binary (TCS BIN), a bit-oriented protocol characterizes the call control motioning for the foundation of speech and data calls between Bluetooth gadgets. Wireless application protocol (WAP) in the Wireless Application Environment (WAE) decides the mode of the transformation such as User Datagram Protocol (UDP) and Transmission Control Protocol (TCP). Frame formatting is one of the most vital tasks for relentless data transmission is shown in Fig. 9. There are mainly three primal parts named as access code, header and payload. Access Code (AC) consists of a preamble, synchronization and tailer where each of them conquered 4 bits. It serves the purpose of packet identification when every channel exchanged on the channel is proceded by AC. Meanwhile, piconet recipients compare the incoming signal with AC. If the bits are mismatched, the received packet is considered invalid and the rest of the contents are being ignored. Another purpose of using AC is for synchronizing and offset compensation. Threelayer of a combination of a packet involving active mode address, type, flow, acknowledgment, sequential number and header error check make the header packet is shown in Fig. 10. Henceforth, AM_ADDR contains the address of one of the slaves. Meanwhile, TYPE, identifying the packet type requires 4 bits. FLOW, ARQN and SEQN require 1 bit each represented flowcontrol, acknowledgment and sequence numbering scheme for packet ordering accordingly.
The most significant 8 bits performed error detection for header integrity. Finally, a header, the payload data and a Cyclic Redundancy Check (CRC) make the payload packet, is shown in Fig. 11, where header and CRC have used as least significant bits and most significant bit accordingly. Figure 12 shows the low energy data transmission model used in the robotic system for manipulating the writing hand. Henceforth, the android phone starts taking input from the user for data transmission to control the robot with a short distance. In the logical link control and adaptation layer, the logical is established as well as the data flow is controlled. Eventually, the L2CAP sends and receives packets to applications through the logical channels and each logical channel is classified by a Connection Identifier (CID). Then, in the Link Manager (LM) layer, the session is transformed for the physical link management. Finally, with the help of radiofrequency, the clients send the bits to the receiver in the physical layer according to the frame formatting operation.

Flowchart of the Robot Program
After the android sends a data string to the Arduino through a Bluetooth device. The Arduino receives the string as an input. This string is first split at. This operation separates the three information including the position coordinate and operation mode. They are then converted to the integer data type. If the Arduino receives a null byte, then it commands the pen servo to lift the pen and also sets the count variable to zero.
On the other hand, if the received data bytes are not null, the Arduino first selects the operation mode. The flowchart of the data received and the mode selection is illustrated in Fig. 13. In the writing mode, the system does not plot anything at the first time. Rather it positions its servos for plotting during this time. It is for avoiding the slip of the pen while writing. The flow chart of the writing mode is shown in Fig. 14. The initial value of the count variable is zero. Hence, the system does not drop the pen but performs some calculation and positions the angle and radius servo.
At the end of the process, it increases the variable count by one. From the second iteration, the value of the variable count is no longer zero. Therefore, the system drops the pen and starts to write. In the case of erase mode, the operation is exactly the same as the writing mode as in Fig. 15. As the eraser is three centimeters behind the pen so while calculating the radius, the program adds three centimeters with the radius. After completing both writing and erase mode the count variable again sets to zero.

Output Results of the System
At the beginning of the drawing process, the robot moves towards the starting position. This time the marker pen is uplifted. When the robot reaches that starting position, it moves the pen vertically until the pen touches the drawing plane. Then it starts to plot the trajectory of the user's finger. Figure 16 displays the input drawing and the output drawing of the system. Here, it draws a character "x". The results of drawing one of the four basic drawing strokes are manifested in Fig. 17. The left column of each pair signifies the drawing on the android screen and the right column denotes the output of the system. In the case of the vertical line, the output is satisfactory. It is much identical with the input on the Android screen. For the horizontal line, the robotic system outputs a slightly declining line but the drawing is recognizable. While drawing an almost circular shape, the result is not good enough. For making an angle, the left falling line is much good but the same slightly declining horizontal line is notable. By analyzing the output results, it seems that the outputs are not exactly identical but the drawing shapes are easily recognizable.
Zigzag of the line is noticed in every drawing. It is due to the limitation of writing a fractional value of angle. Again, The nearer the drawings to the origin, the better the output results. The response of the robotic system is slower than the user's finger movement on the screen. Thus, a slower finger movement produces a good output result.

Conclusion
This paper exhibits a human finger supervised trajectory approach to robotic drawing. To demonstrate the human trajectory, the patterns of four basic strokes are used. An android app is developed to take users drawing. Scaling function, CAD design and inverse kinematics calculations are performed to achieve robot joint values. The experimental outcomes show that using a human finger trajectory on the android screen can conveniently reconstruct the drawing. The robotic system can efficiently draw many varieties of strokes without using any complex processing. The modified rope pulley mechanism shows promises while performing the job of the 2 DOF planner robotic arm. The integration of Internet of Things (IoT) based polar writing arm would be the possible future direction of the proposed work.