Reference / Paper · 2022
US Patent 11,314,339 B2 — Sinden, Control Device for Detection (2022)
Sinden's border-tracking patent.
- sinden
- border tracking
- camera gun
Reference / Paper · 2022
Sinden's border-tracking patent.
US011314339B2 ( 12) United States Patent ( 10) Patent No .: US 11,314,339 B2 Apr. 26, 2022 (45 ) Date of Patent : Sinden ( 54 ) CONTROL DEVICE FOR DETECTION ( 58 ) Field of Classification Search CPC G06F 3/0325 ; G06F 3/0346 ; G06F 3/0383 ; A63F 13/219 ; A63F 13/426 See application file for complete search history . ( 71 ) Applicant: SINDEN TECHNOLOGY LTD , Coventry ( GB ) U.S. PATENT DOCUMENTS Coventry ( GB ) 2001/0010514 Al 2005/0104849 Al Subject to any disclaimer, the term of this patent is extended or adjusted under 35 U.S.C. 154 ( b ) by 0 days. ( * ) Notice: (21 ) Appl. No .: FOREIGN PATENT DOCUMENTS 17/054,958 GB JP International Search Report and Written Opinion dated Jul. 26 , 2019 Primary Examiner Christopher E Leiby (74 ) Attorney, Agent, or Firm - Maier & Maier , PLLC ( 57 ) May 13 , 2018 ( GB ) Oct. 1 , 2018 ( GB ) CPC 1807749 1815961 coordinates of the received image signal that define at least a part of an outline of the display; calculating , based on the array of pixel coordinates and at least one centre pixel of the received image signal, an orientation of the image detector in relation to the display. An apparatus for detecting a display is also provided . The apparatus is configured to carry out the method and includes an image detector and an image processor configured to be in wired or wireless communi cation with a display. 17 Claims , 8 Drawing Sheets ( 2006.01 ) ( 2014.01 ) ( Continued ) (52) U.S. Cl . G06F 3/0325 (2013.01 ) ; A63F 13/219 (2014.09 ) ; A63F 13/426 ( 2014.09 ) ; G06F 370346 (2013.01 ) ; G06F 3/0383 (2013.01 ) 13 ABSTRACT A method for detecting a display is provided , the method including receiving an image signal from an image detector ; detecting, by an image processor, that the received image signal includes an image of at least a portion of a display; determining, by the image processor, an array of pixel Foreign Application Priority Data G06F 3/03 A63F 13/219 10/2008 2/2003 of corresponding application No. PCT/GB2019 /051303 ; 14 pgs . (Continued ) Prior Publication Data US 2021/0247852 A1 Aug. 12 , 2021 ( 51 ) Int . Cl . 2448034 A 2003038849 A OTHER PUBLICATIONS PCT Pub . Date : Nov. 21 , 2019 ( 30 ) 8/2001 Ishino 5/2005 Hoile (Continued ) May 13 , 2019 (22) PCT Filed : PCT/GB2019/ 051303 ( 86 ) PCT No .: $ 371 (c ) ( 1 ) , Nov. 12 , 2020 ( 2 ) Date : ( 87 ) PCT Pub . No .: WO2019 /220086 ( 65 ) References Cited ( 56 ) (72 ) Inventor : Andrew James Sinden , Coventry ( GB ) ( 73 ) Assignee : SINDEN TECHNOLOGY LTD , 14 22 20 -51 o 10 19 -22 16 211 -29 25 22 US 11,314,339 B2 Page 2 ( 51 ) Int. Ci. A63F 13/426 G06F 3/0346 G06F 3/038 ( 56 ) ( 2014.01) ( 2013.01 ) ( 2013.01 ) References Cited U.S. PATENT DOCUMENTS 5/2005 Cheng et al . 2005/0107160 A1 2006/0023111 A1 * 2/2006 DeMenthon 2006/0176274 A1 2009/0046061 A1 8/2006 Cheng et al . 2009/0153479 A1 2011/0124410 A1 * 2014/0139455 A1 * 2014/0191957 Al G06F 3/0346 348/360 2/2009 Denoue et al . 6/2009 Gu et al . 5/2011 Mao 5/2014 Argiro 7/2014 Cheng A63F 13/42 463/31 G06F 3/0346 345/173 OTHER PUBLICATIONS British Combined Search and Examination Report dated Nov. 28 , 2018 of corresponding application No. GB1815961.6 ; 6 pgs . * cited by examiner U.S. Patent Apr. 26, 2022 US 11,314,339 B2 Sheet 1 of 8 14 13 22 1 20 51 10 19 16 22 21 Fig . 1 -29 25 22 52 50 12 20 16 25 15 s 23 21 29 Fig . 2a 50 16 22 27 15 22 12 22 23 18 22 20 25 21 22 Fig . 2b 29 U.S. Patent Apr. 26 , 2022 US 11,314,339 B2 Sheet 2 of 8 14 17 13 Fig . ?? 17 13 28 14 Fig . 3b U.S. Patent Apr. 26 , 2022 30 32 14 28 US 11,314,339 B2 Sheet 3 of 8 13 30 30 30 Fig . 30 +] 34 34 36 Fig . 3d U.S. Patent Apr. 26, 2022 1 I x= 0 , y= 0 x = 160 0 %x -y = 90 1 x = 480 x = 320 ¡60 % ish x 100 % x X axis x =640 ,y = 0 0% y 40 -y= 180 US 11,314,339 B2 Sheet 4 of 8 38 34 80 % ish y 34 100 % 36 -y = 270 y axis x= 0 , y= 360 x =640 , y = 360 Fig . 3e 30 30 44 42 -36 30 30 32 Fig . 4 U.S. Patent Apr. 26 , 2022 Sheet 5 of 8 US 11,314,339 B2 14 Fig . 5a Fig . 5b Fig . 5c Fig . 5d Fig . 5e Fig . 5f Fig . 5g Fig . 5h 14 13 64 12 12 62 Fig . 6 U.S. Patent Apr. 26 , 2022. 100 US 11,314,339 B2 Sheet 6 of 8 O 104 102 Fig . 7A +104 100 102 106 Fig . 7B -104 100 ? 102 106 Fig . 7C U.S. Patent Apr. 26, 2022 US 11,314,339 B2 Sheet 7 of 8 102 -106 100 104 Fig . 7D 102 106 104 100 Fig . 7E 110 Q 102 106 -104 .. 100 110 110 Fig . 7F U.S. Patent Apr. 26 , 2022 Sheet 8 of 8 US 11,314,339 B2 -104 100 -102 106 Fig . 8A O -104 -106 -108 Fig . 8B US 11,314,339 B2 1 2 CONTROL DEVICE FOR DETECTION calculating , based on the array of pixel coordinates and at BACKGROUND display area of the display, wherein the applied artefact may be configured to aid the detection of the display, least one centre pixel of the received image signal, an FIELD orientation of the image detection means in relation to the display. This invention relates to improvements in or relating to a 5 According to another aspect of the invention , there is method for detecting a display and an apparatus for carrying provided an apparatus configured to detect a display accord out the method , the apparatus comprising a control device ing to the method of the present invention , the apparatus and, in particular, the invention relates to a method for comprising: an image detection means ; and detecting a display device using a line of sight control device as a controller and an apparatus for carrying out the method 10 an image processor configured to be in wired or wireless comprising a line of sight control device to detect its communication with a display. alignment relative to a display device . In addition, the In some embodiments, the apparatus for detecting a present invention relates to a line of sight control device to display comprises: a control device , detect the display screen . 15 an artefact being applied to a display content within a It is commonly known that some controllers can be used an image detection means configured to capture an image to interact with objects on a display screen . A user often uses comprising the display ; and a controller for many purposes. For example; for providing 20 an image processor for receiving a first signal from the an indicator on a screen where the user is pointing; for image detection means , the image processor is config detecting objects on a screen for entertainment purposes ured to process the captured image to determine an such as playing video games ; and / or for using a controller array of image pixel coordinates of the display which is for training and educational purposes. Unfortunately, many compared to an image pixel coordinates of the centre of controllers used for detection can be inaccurate and do not 25 the captured image , and truly reflect the position of where the user using the conwherein the image processor further determines a pointing alignment of the control device in relation to the troller points to on a display screen . For gaming purposes, the controller can be used as a hand -held controller device display on the basis of the determined image pixel coordinates to detect the display. or a gaming accessory such as a gun - style accessory . Many users in the past have enjoyed using lightgun technology for 30 According to another aspect of the present invention , there is provided a computer -readable medium comprising playing shooting games. However, classic lightgun technology does not work on instructions which, when executed by a processor, cause the modem displays such as LCD/ LED televisions or LCD processor to carry out the methods described herein . projectors. Classic light guns use the fast line by line refresh In some embodiments, the applied artefact can be a of the cathode ray tube ( CRT) that has precise timing. 35 background, foreground, a border or a partial border adapted Modem LCD /LED televisions or LCD projectors process to partially surround or completely surround the display the image as a complete image and then refresh the display content within the display area . The artefact may be partially as a complete frame usually at a slower rate than a CRT. As transparent. The display area may be a display screen of the a result , users who play classic lightgun computer games are display. not able to play with a classic lightgun on a modem 40 In some embodiments, the background, foreground, par television . tial border or the border may be applied to / around at least The current alternatives are using a mouse , keyboard and one edge or corner of the display area . In some embodi joystick to move the cursor or for a gun style solution putting ments, the applied artefact may be a rectangular or square some extra hardware such as infrared LEDs next to the border around the inner edge of the display content to assist television that can be detected at the front of the remote / gun 45 the detection of the display area . to deduce approximately where the remote /gun is pointing In some embodiments the artefact may be based on an on the display screen . This does not provide true line of sight algorithm that takes into account pixel attributes such as , but response and provides a less accurate way to calculate where not limited to , pixel location , colour and brightness. The the control device is pointing at the display, unless the algorithm will then appropriately change pixel values such 50 as , but not limited to , colour and brightness with the purpose system goes through a calibration routine. However, a calibration routine is often aa time consuming of making the image recognition process easier and more process because a user must go through a calibration routine accurate . every time the user changes their position , angle and / or The apparatus may further comprise the application of changes their equipment. additional artefacts around at least one edge or corner of the It is against this background that the invention has arisen . 55 display content . According to another aspect of the present invention there is provided, an apparatus for detecting a display device , the apparatus comprising: According to an aspect of the present invention , there is a control device, provided a method for detecting a display, the method 60 an artefact being applied to a display content within a comprising the steps of: display area of the display, wherein the applied artefact receiving an image signal from an image detection means ; may be configured to aid the detection of the display, detecting, by an image processor, that the received image an image detection means configured to capture an image signal comprises an image of at least a portion of a display; comprising the display ; and determining, by the image processor, an array of pixel 65 an image processor for receiving a first signal from the coordinates of the received image signal that define at least image detection means , the image processor is config ured to process the captured image to determine an a part of an outline of the display ; SUMMARY US 11,314,339 B2 3 4 array of image pixel coordinates of the display which is from the image processor and may send an output signal to compared to an image pixel coordinates of the centre of the display device to display the pointing alignment/position the captured image , of the control device on the display. wherein the image processor further determines a pointing The indicator may be a computer mouse cursor or a alignment of the control device in relation to the 5 targeting crosshair used to display the pointing alignment/ display on the basis of the determined image pixel position of the controller on the display such as a TV screen , coordinates to detect the display. laptop screen , a computer screen , a projector screen , a The apparatus of the present invention may be configured acinematic screen , mobile phone screen , tablets, or on any to detect , capture , collect multiple signals, process the other electronic screens . captured image and analyse the captured image in real time 10 This can be advantageous the indicator can be to determine where the pointing alignment, position, angle used to accurately show wherebecause a user handling the control and distance of the control device is in relation to the display. device is pointing to on the screen . The may be As the control device can be directly used for detecting the programmable to enable it to automaticallyindicator and accurately display , even if a user handling the control device changes their operating alignment/position, distance from the display 15 the showscreen where. aInuser handling control device is pointing to on some alternative embodiments , the control or angle to the display the control device does not require calibration and recalibration . Thus, the apparatus of the device may comprise an indicator for displaying the pointing present invention does not require a calibration routine when alignment /position on the display device . in use , which can significantly improve the control device In some embodiments , an indicator may not be required. for detecting, processing and analysing the image, and a user 20 This may give a different user experience and the user may experience handling the control device. In another advan- rely on the line of sight accuracy to know where they are tage , the apparatus of present invention does not require any pointing on the display. external hardware for detecting the relative location of the In some embodiments, the indicator may be a tracking display device , such as an infra -red LED array next to /or indicator such as a tracking cursor which can continually 25 track and display the pointing alignment or position of the around the display device . In addition , the present invention provides an advantage control device on the display screen . in that it can be used detect modem technology display In some embodiments, the apparatus of the present inven devices such as LCD or LED , television screens , computer tion mayfurther comprise the control device having a trigger monitors , modem projector displays, mobile phone devices, input associated with a processor, the trigger input may be 30 adapted to be triggered by a user which may be configured tablets , cinematic screens or gaming devices . It is also an advantage that the apparatus of the present for operating a computer programme, thereby to enable the invention may be able to provide a more accurate way to processor to send aa further signal to an electronic device, the detect an object in the captured image , such as the display electronic device may be configured to respond to the trigger input and perform a plurality of outputs . device . The control device may be a line of sight control device . 35 In some embodiments, the captured image can be pro The control device may be a hand -held device such as a cessed by the image processor to remove background noise controller. The controller may be mobile and be operable by from the image gathered by the image detection means . This a user's left hand , right hand or both . The line of sight may provide a clearer image for image processing . In some controller may comprise a suitable configuration or it may cases , the image processor may remove background noise to comprise a strap for the user's hand to grip the controller to 40 provide a higher quality image for analysis . operate the line of sight controller freely for example, the In some embodiments, the captured image is processed by user can rotate the controller, move the controller from side the image processor to enhance the image gathered by the image detection means . The enhanced image may provide a to side and / or up or down . In some alternative embodiments, the image processor higher quality image which may make it easier for the image may further determine the pointing alignment, position, 45 processor to recognise the image and process the captured angle and distance of the control device in relation to the image more efficiently. In some embodiments, the captured image could be manipulated to enhance the characteristics of display based on the image pixel coordinates. Processing the captured image using the image processor, the display in the captured image . which may be a local image processor, reduces the transIn some embodiments , the image detection means may be mission time for sending the processed data, and it may 50 suitable for blob detection , edge detection , corner detection , reduce the amount of hardware on an electronic device being quadrilateral detection , rectangle or square detection in the controlled . The image processor may be configured to image . process multiple signals received from the image detections In some embodiments, the processed image may be means in real time to determine the array of image pixel analysed /processed by the image processor to determine the coordinates of the display. In some embodiments, processing 55 array of coordinates of the display on the image based on of the captured image can be performed by the image one or more of the following: contour size , position , inten processor on board the control device or the raw image/ sity, colour, brightness, velocity , shape, outline, edge , angle , video input can be fed to an electronic device and be light frequency, light polarisation, flickering, refresh rate , processed by the electronic device , such as a computer or a contrast and / or size of the display. computer - implemented device . This reduces the amount of 60 Preferably, the captured image may comprise a substan hardware required and the complexity of the hardware tially rectangular -shaped or a substantially square -shaped display. A rectangular - shaped or square - shaped display required in the control device . In some embodiments, the apparatus may further com- device may comprise substantially sharp edges which may prise an electronic device which may be configured to mean that the image detection means can easily detect the provide an indicator to display the pointing alignment, 65 display on the captured image . position , angle or distance of the control device on the As the user may not be directly in front of the display, a display . The electronic device may receive a second signal rectangular or square display may appear to be a distorted US 11,314,339 B2 5 6 quadrilateral on the captured image . The image processing screen may typically be a modern TV screen . In some may take into account this distortion when calculating a embodiments, the TV screen may be aa LCD TV screen or an pointing location or position . LED TV screen. The captured image may comprise a display having an An illuminated display device can be advantageous outer rectangular shape in order to allow the image proces- 5 because it may enable the image detection means to detect sor to determine the array of image pixel coordinates of the and capture a clear image comprising the display. This may display consistently with high accuracy . then allow the image processor to easily recognise and In some embodiments, the captured rectangular- shaped or the square - shaped display may be analysed to determine an process the captured image comprising the display more efficiently . The image process can exclude pixels that do not array of coordinates based on at least one corner or one or 10 match a minimum brightness. The brightness can be used as two edges of the display. The array of coordinates may be part of aa noise removal process during image processing. based on more than two or more different corners or edges Optionally, the shape, outline , size , contour level , gamma, of the display. Preferably, the image processor may be used angle , light polarisation , brightness, contrast, flickering, to determine the array of coordinates based on four corners refresh rate , colour and / or size of the display may be useful or edges of the rectangular -shaped display . 15 to enable the image detection means to detect and capture a In some embodiments, the image processor may process clear image that can be easily processed. the captured image to determine where the centre pixel In some embodiments , the apparatus may further com coordinates in the image are in relation to the array of image prise a storage media for storing captured images , the array pixel coordinates of the display. Preferably, the determined of coordinates of the display and / or the coordinates of the display coordinates being pointed at by the control device 20 pointing alignment/ position of the control device. are represented as X horizontal percentage and Y vertical In another aspect of the invention there is provided , a percentage. method for detecting a display using an apparatus according Optionally, the determined coordinates are represented as to a previous aspect of the present invention , the method a ratio of X horizontal percentage and Y vertical percentage . comprising the steps of: By using X horizontal percentage and Y vertical percent- 25 ( i ) capturing an image comprising the display using an age as coordinates , the image processor may be able to image detection means ; process and analyse the captured image with the display ( ii ) an image processor receiving a first signal from the being at low resolution and /or at high resolution . In other image detection means ; and words, the resolution of the display image is irrelevant. ( iii ) processing the captured image using the image pro In other embodiments, the determined coordinates may be 30 cessor image to determine an array of image pixel represented as X horizontal and Y vertical pixel coordinates coordinates of the display which is compared to an if the resolution of the display is known or X horizontal and image pixel coordinates of the centre of the captured Y vertical distance coordinates if the size of the display is image ; known. wherein the image processor further determines a pointing In some embodiments, the apparatus may further com- 35 alignment of the control device in relation to the display on prise at least one communication interface such as a USB the basis of the determined image pixel coordinates. interface, HDMI interface, a WIFI interface or Bluetooth In another aspect of the invention , there is provided a wireless interface, wherein the communication interface method for detecting a display using an apparatus described may be configured to connect the control device to the image above , the method comprising the steps of: detection means , the image processor, the electronic device 40 ( i ) capturing an image comprising the display using an and / or a display device . image detection means ; In some embodiments, the method may further comprise ( ii ) applying an artefact to a display content within a applying an image artefact, such as a white border or display area of the display, an artefact being applied to background, which can be applied to show on the display. a display content within a display area of the display, The image artefact may also be partially transparent. Apply- 45 wherein the applied artefact is configured to aid the ing an image artefact to the display screen can optimise the detection of the display, accuracy of the controller and may also help reduce pro( iii ) receiving , via an image processor, a first signal from cessing requirements. The image artefacts can be applied the image detection means ; and around the edge , on top of or in the background of the ( iv ) processing the captured image using the image pro displayed content to make display recognition and process- 50 cessor image to determine an array of image pixel ing easier for the processor. The image artefact may be any coordinates of the display which is compared to an change to the display content that enhances the outline or the image pixel coordinates of the centre of the captured image; shape of the display to make display recognition and pro cessing easier for the processor such as during image wherein the image processor further determines a pointing recognition processes. 55 alignment of the control device in relation to the display on In some embodiments, the apparatus may further com- the basis of the determined image pixel coordinates to detect prise at least one power unit in the control device connected the display. to a power source, which could be used to provide power to Preferably, the method may comprise the step of adding the image detection device and / or the image processor. a rectangular or a square border around the inner edge of the The image detection means may be a digital camera or it 60 display content to assist the detection of the display area . may be any electro -magnetic radiation image detection This may enhance the outline and shape of the display which means . In some embodiments , the image detection means may be a video recorder, or a mobile phone device, or a webcam or any other devices that can be used for detecting can be used during the image recognition process to identify In some embodiments , the method may further comprise the display. an image . 65 the step of displaying the pointing alignment/position of the In some embodiments, the display device can be a TV control device on the display using an electronic device , screen , or a projector screen or a cinematic screen . The TV which may be configured to provide an indicator. The US 11,314,339 B2 7 8 electronic device may receive a second signal from the wherein the image processor further determines a pointing image processor and may send an output signal to display alignment of the control device in relation to the display on the pointing position of the control device on the display. the basis of the determined image pixel coordinates. In some embodiments, determining the array of coordiIn some embodiments, the control device may further nates of the display may be based on contour size , position , 5 comprise a trigger input associated with a processor, the intensity , colour, flickering, refresh rate , light frequency, trigger input can be adapted to be triggered by a user which brightness, light polarisation, velocity, shape , outline , con- may be configured for operating a computer programme, trast, angle or size of the display. In some embodiments, the thereby to enable the processor to send a further signal to the user may change any one of following based on contour size , electronic device , the electronic device may be configured to position , intensity, light frequency, light polarisation, colour, 10 respond to the trigger input and perform a plurality of flickering, refresh rate , brightness, velocity , shape, contrast, outputs. In some embodiments, the control device may comprise the electronic device. display . In some embodiments, the method may further In some embodiments, the control device may further comprise the step of applying an image artefact to the comprise aa vibration motor, speaker or a solenoid intercon display. 15 nected to or in association with the trigger input and / or In some embodiments , the method may further comprise additional inputs, and / or triggered by the electronic device , storing the captured image, the array of coordinates of the wherein the vibration motor, solenoid or speaker may be display device and / or the coordinates of the pointing align- configured to provide feedback upon the trigger sensor being ment/position of the control device onto a storage media . triggered by a user . As an example, the vibration motor can In some embodiments, the method may further comprise 20 be used to provide haptic feedback . As another example, the the step of calculating a changing distance, referred to as a solenoid can provide a recoil effect. Z axis , from the control device to the display. The Z axis can In some embodiments, the control device may further be calculated as the user operating the control device moves comprise an additional button and / or a scrolling wheel or an closer to the display or further away from the display. additional control or any other input controls. These addi In some embodiments, the method may further comprise 25 tional buttons and scrolling wheel may provide a plurality of the step of calculating an angle of the pointing alignment/ different functionality for the user controlling the control position of the control device to the display screen . device . For example, the scrolling wheel may be useful to In some embodiments, the method of the present inven- activate to scroll the content that may be displayed on the tion may further comprise validating the captured image display device. The additional button may be used to control comprising the display by a further image processor, 30 the image detection means . An additional control may be wherein the array of image pixel coordinates processed by used to control the image artefacts added to the display the further processor based on one or more of the following: content, for example changing one or more of the following contour size , position , intensity, colour, flickering, refresh but not limited to the thickness, size , shapes or colour of the rate , light frequency, brightness, velocity, shape, outline, border. edge, angle , contrast, polarisation , and / or size of the display 35 In another aspect of the invention , there is provided a kit is validated by matching the determined image pixel coor- for detecting a display according to any of the previous dinates determined by the image processor based on contour aspect of the invention, the kit comprising: size , position, intensity, colour, flickering, refresh rate, light a image detection means for capturing an image ; and frequency, brightness, velocity, shape , outline , edge , angle, an image processor configured to process the captured 40 image . contrast, light polarisation and / or size of the display. In a further aspect of the invention, there is provided a In some embodiments , the display can be a television, a control device for detecting a display, the control device laptop screen , a computer screen , a projector screen , a comprising : cinematic screen , mobile phone screen, tablet, or on any an image detection means configured to capture an image other electronic screens , wherein the display is a modem angle or size of the display to optimise the detection of the comprising the display, an artefact applied to a display content within a display 45 technology display screen such as an LCD or LED screen . In some embodiments, a border could be added around the area; and display content using a physical modification such as an an image processor for receiving a first signal from the illuminated neon strip . This physical method would still image detection means, the image processor is config- utilise the shape detection and brightness detection of the ured to process the captured image to determine an 50 software method but would not require the ability to adapt array of image pixel coordinates of the display which is the display content. This would be useful for example on an compared to an image pixel coordinates of the centre of old arcade gaming machine where it is not possible to the captured image, modify the display content. wherein the image processor further determines a pointing alignment of the control device in relation to the display on 55 BRIEF DESCRIPTION OF THE FIGURES image detection means, the image processor is configured to process the captured image to determine an ing to FIG. 1 , FIG . 2b , shows a cross sectional view of the control compared to a centre image pixel coordinates in the captured image , FIG . 3b shows a captured image comprising a display, the basis of the determined image pixel coordinates. In another aspect of the invention, there may be provided , The invention will now be further and more particularly a control device for detecting a display device , the control described, by way of example only, and with reference to the accompanying drawings, in which: device comprising : an image detection means configured to capture an image 60 FIG . 1 shows an apparatus according to an aspect of the comprising the display ; and present invention , an image processor for receiving a first signal from the FIG . 2a , provides a side view of a control device accord array of image pixel coordinates of the display which is 65 device according to FIG . 2a , FIG . 3a shows a captured image comprising a display, US 11,314,339 B2 10 9 FIG . 3c shows a captured image comprising a display, on an electronic device 20 for example a computer device. FIG . 3d shows an array of plotted coordinates of the Alternatively, the image processor may be built within the display , control device and / or the camera . FIG . 3e shows an array of plotted coordinates of the Referring to FIGS . 2a and 2b , there is shown an example display , 5 of a control device 12. The control device may be a remote FIG . 4 provides an illustration of a detecting a display at controller or it may be a gaming accessory device , such as a gun accessory as illustrated or modified game controller. In an angle according to FIGS . 1a , 1b , 2 , 3a to 3e , FIG . 5a shows the display device at different distance other examples, not illustrated in these figures it may be a mobile phone device or a computer. The control device may and / or angles according to FIGS . 1 to 4 ; FIG . 55 shows the display device at different distance 10 be used to calculate and provide feedback where the point ing alignment/position is at the display 13 , as shown in FIG . and / or angles according to FIGS . 1 to 4 ; FIG . 5c shows the display device at different distance 1. The control device 12 may operate at a distance from the display or the control device may operate at an angle to and / or angles according to FIGS . 1 to 4 ; FIG . 5d shows the display device at different distance detect aa distorted shape comprising the display, as shown in 15 FIG. 4 . and / or angles according to FIGS . 1 to 4 ; FIG . 5e shows the display device at different distance The control device may have an aperture 15 in which the image detections means 16 may be positioned in such a way and / or angles according to FIGS . 1 to 4 ; FIG . 5f shows the display device at different distance to enable the lens of the image detection means 16 to capture an image . In some embodiments, the control device may and / or angles according to FIGS . 1 to 4 ; FIG . 5g shows the display device at different distance 20 have more than one aperture. In some embodiments, a plurality of image detection means may be provided. The and / or angles according to FIGS . 1 to 4 ; FIG . 5h shows the display device at different distance camera 16 or any other image detection means may be built into the controller 12. Alternatively, the camera 16 may be and / or angles according to FIGS . 1 to 4 ; FIG . 6 shows an upward and downward direction of the mounted or attached on top of a surface of the control device 25 12. As the camera may be mounted onto the control device , control device; the centre of the camera would point in line with a user's line FIG . 7a shows a display with no border ; FIG . 7b shows a display with an image artefact added to of sight. Where the camera 16 points to may be equal to the pointing direction 19 of the control device 12. The camera the outline/edges / corner of a display content; FIG . 7c shows a display with an image artefact added to then records an image comprising the display as shown in 30 FIGS . 3a and 3b . The camera may take a 1D , 2D or 3D the outline/ edges / corner of a display content; FIG . 7d shows aa display with an image artefact added to image. The camera may take an image at any resolution e.g. at the outline/ edges /corner of a display content; FIG . 7e shows a display with an image artefact added to high or at low resolution. The image detection means may be connected to an electronic device 20 e.g. a computer using the outline/edges /corner of a display content; FIG . 7f shows a display with an image artefact added to 35 a communication interface such as through a USB cable or a HDMI cable 22. Furthermore, the camera or the control the outline/ edges /corner of a display content; FIG . 8a shows a representation of a light polarisation device may be connected to the electronic device 20 via filter; and Bluetooth or any other wireless communication means . For FIG . 8b shows a representation of a light polarisation example, the wireless communication may utilise a wireless 40 internet connection such as WIFI or 1G , 2G , 3G , 4G , 5G and beyond . As shown in FIGS . 1 , 2a and 2b , there is also shown a primary select control or a trigger input 21. The trigger input is interconnected with a trigger sensor/ button 27 which is filter. DETAILED DESCRIPTION OF THE EMBODIMENTS As used herein , unless otherwise stated, the image pro- 45 interconnected to the image processor 18 via the cable 22 . cessor may be defined as a processor which is processing the The trigger input 21 can be associated with a processor 18 . captured images and / or it may include processing user Furthermore , the trigger input 21 can be adapted to be inputs such as buttons and triggers and / or sending commu- triggered by a user which may be configured for operating nication signals to an electronic device comprising data a computer programme, thereby to enable the processor 18 relevant to the process e.g. captured images or calculated 50 to send a further signal to the electronic device 20. The coordinates, or it may provide any other processing means electronic device may then be configured to respond to the trigger input and perform a plurality of outputs. required on the device . Referring to FIGS . 1 , 2a and 2b , there is provided an In other embodiments not shown in the accompanying apparatus 10 for detecting a display device 14 such as a drawings, the plurality of outputs may be configured to television . The apparatus 10 comprise a control device 12 , 55 perform a variety of functionality including, but is not an image detection means 16 , which may be configured to limited to , being triggered by the user to perform one or > capture an image comprising the display 13 and an image more of the following; a mouse click /movement, keyboard processor 18 for receiving a first signal from the image button press or joystick movement/button press . detection means 16. The image detection means 16 may be The controller can have further additional controls 23 that a digital camera or a webcam which can be used to take 60 may be operable by a user. The additional control 23 may be images or videos which comprises the display device 14 . an operable control such a secondary trigger input, buttons Alternatively, the image detection means 16 may be used to or scroll wheels in which a particular functionality can be detect aa live image . The image processor 18 may receive a assigned /mapped towards providing the user benefit in the first signal of the captured image from the image detection computer programme being operated. As an example, the means 16 such as a camera . The image captured by the 65 secondary trigger input 23 may be associated with a mapped camera may then be processed by the image processor 18 . functionality that is similar to the function of aa right -mouse The image processor may be in a form of a CPU /master unit click , when operable by the user. The controller may be a US 11,314,339 B2 11 hand -held device. The controller can have a suitable surface 12 television or only to move the cursor when then the trigger 25 to allow a user to hold / grip the