Unity implements the mouse to pick up the pixel color of the specified area of the computer screen

Hits: 0

Article directory

👉1. Preface and knowledge points

1 Introduction

[During development, we often encounter the need to dynamically modify the color of objects or UI, and we need to pop up a color picker] like the Unity editor to select a color.
I thought that if I had to develop a color picker from 0, it would be a lot of work. In the end, I chose to use the third-party color picker plugin ColorPicker to complete the color picker function. But in the process of using the ColorPicker plug-in, I found that when picking up the pixels of the computer screen with the color picker, an error will be reported: attempting to ReadPixels outside of RenderTexture bounds! Reading (2396, 2905, 2412, 2921) from (1920, 1080) (meaning trying to The read pixel exceeds the [texture] from (1920, 1080)** (meaning trying to The read pixel exceeds the [texture] boundary)
to check the reason for the error:
The original logic of the author of the ColorPicker plug-in to write a color pen to pick up color is to intercept the image of the specified area of ​​the mouse coordinate point, and then complete it by obtaining the pixel color at the center point of the image. Take color. Since the plugin source code uses the Texture2D.ReadPixels method to draw texture images, this method can only read the pixels within the Game window range of the current resolution. Once it exceeds this range, it cannot be read and an error will be reported.
Solution: I didn’t find a way to read the pixels of the computer screen outside the Game window in the Unity library, so I turned the direction to the .NET class library. You can use System.Drawing.dll in the .net library to take screenshots of the computer screen, create bitmap data of the image, convert it into a byte array, load the byte array into Unity’s Texture Image, and finally read the Texture The pixel color of the texture.

2. Knowledge points

1. Use the .Net class library System.Drawing to capture the image of the area specified by the mouse
2. Convert the bitmap data Bitmap to a byte array
3. Convert the byte data to Unity’s Texture texture image
4. Through Unity’s Texture.GetPixel() Method to get the pixel color of the specified point on the image

👉2. Realize the mouse to pick up the pixel color of the specified area of ​​the computer screen

1. Preparation

Import the .NET class library System.Drawing.dll. (Use the search tool to find or find the System.Drawing.dll under the current Unity running platform from your Unity installation path)

2. Use the .Net class library System.Drawing to capture the image and convert it to the texture image Texture supported by Unity

New core script GetScreenPixel.cs:

//Reference dependent namespace 
using UnityEngine;
 using System.Drawing;
 using System;
 using System.Drawing.Imaging;
 //Class for getting screen pixels 
public  class  GetScreenPixel
{
    private  static Bitmap bitmapSrc; //The bitmap data of the screenshot 
    private  static  int multiple; //The scale factor of the screenshot, which can be used to zoom in and out

    ///  <summary> 
    /// Take a screenshot of the mouse point and convert it to Unity's Texture2D texture image 
    ///  </summary> 
    ///  <param > </param> 
    ///  <param > </param> 
    ///  <returns> </returns> 
    public  static Texture2D GetTexture ( int width, int height )
    {
        Size size = new Size(width, height); //Intercepted size 
        bitmapSrc = new Bitmap(width, height); //Acquired bitmap size 
        multiple = 1 ;
        BitmapReset(bitmapSrc); //Reset the picture to solve the residual BUG of the image beyond the screen part 
        System.Drawing.Graphics g = System.Drawing.Graphics.FromImage(bitmapSrc); //Create a new image based on the bitmap data 
        g.CopyFromScreen( new Point(System.Windows.Forms.Cursor.Position.X - width / ( 2 * multiple), System.Windows.Forms.Cursor.Position.Y - height / ( 2 * multiple)), new Point( 0 , 0 ) , size); //Transfer the image data of the specified area size from the screen to Graphics to draw it
        IntPtr dc1 = g.GetHdc();
        g.ReleaseHdc(dc1); //Release the current handle 
        Texture2D tex = new Texture2D(width, height, TextureFormat.RGB24, false );
        tex.LoadImage(BitmapToByte(bitmapSrc)); //Load image byte array to texture. 
        return tex;
    }
    ///  <summary> 
    /// Convert the bitmap stream to byte stream array 
    ///  </summary> 
    ///  <param > </param> 
    ///  <returns> </returns > 
    public  static  byte [] BitmapToByte ( System.Drawing.Bitmap bitmap )
    {
        // 1. First convert BitMap into memory stream 
        System.IO.MemoryStream ms = new System.IO.MemoryStream();
         //bitmap.Save(ms, System.Drawing.Imaging.ImageFormat.Bmp);//Unity loading bmp format data is not supported 
        bitmap.Save(ms, System.Drawing.Imaging.ImageFormat.Png); //Save the bitmap data as png type data 
        ms.Seek( 0 , System.IO.SeekOrigin.Begin);
         // 2. Convert the memory stream to byte[] and return 
        byte [] bytes = new  byte [ms.Length];
        ms.Read(bytes, 0, bytes.Length);
        ms.Dispose();
        return bytes;
    }

    ///  <summary> 
    /// Reset Bitmap 
    ///  </summary> 
    ///  <param > </param> 
    private  static  void  BitmapReset ( Bitmap bitmap )
    {
        System.Drawing.Imaging.BitmapData bitmapdata = bitmap.LockBits(new Rectangle(new Point(0, 0), bitmap.Size), ImageLockMode.ReadWrite, PixelFormat.Format32bppArgb);
        unsafe
        {
            byte * dataPointer = ( byte *)(bitmapdata.Scan0.ToPointer()); //The address pointer of the data matrix in memory 
            for ( int y = 0 ; y < bitmapdata.Height; y++)
            {
                for (int x = 0; x < bitmapdata.Width; x++)
                {
                    dataPointer[0] = 0;
                    dataPointer[1] = 0;
                    dataPointer[2] = 0;
                    dataPointer[3] = 0;
                    dataPointer += 4;
                }
            }
        }
        bitmap.UnlockBits(bitmapdata);
    }
}

3. Points to pay attention to

  1. When converting bitmap data into byte stream, you need to specify the image format as png or jgp type (the format supported by Unity), otherwise, a red question mark image will be displayed when Unity loads the texture.
  2. C++ pointers are used when processing bitmaps in the GetScreenPixel script. If you are using a version after Unity 2018, please check Allow ‘unsafe’ Code in PlayerSetting/OtherSetting: otherwise, the compiler may report that the unsafe code cannot be recognized. mistake.

👉3. Integrate into the source code of the ColorPicker plug-in to complete the function of the color picker

The core script of the ColorPicker plug-in is ColorPicker. We need to call the GetTexture() method of the GetScreenPixel script in this script to get the screenshot of the mouse point and read the pixel color of the center point of the obtained image. In addition to that, comment out some code.

1. Modify the source code in the ColorPicker script

  1. Add the GetScreenColor method, call this method in the Update function and comment out the original screenshot method

/// The method of getting the mouse point pixel of the computer screen 
        private  void  GetScreenColor ()
        {
            var xCount = m_imageMesh.XAxisCount; //The width of the color picking area 
            var yCount = m_imageMesh.YAxisCount; //The height of the color picking area 
            m_texture = GetScreenPixel.GetTexture(xCount, yCount); //Get the specified width and height of the mouse point color picking area The image 
            m_screenImage.sprite = Sprite.Create(m_texture, new Rect( 0 , 0 , xCount, yCount), Vector2.zero); //The sprite image of the created image is assigned to the color-picking image 
        }

  1. Add OnApplicationFocus(bool focus) method

Because when the mouse is clicked outside the Game window, it does not respond to the mouse click function, so when selecting the off-screen pixel color, we need to use the OnApplicationFocus function, that is, if the mouse is clicked in the background while running Unity, the Unity program will “lose focus” , the OnApplicationFocus function will be called once, and the incoming parameter is False. At this point, we can set the color on the color picker to the pixel color of the center point of the current color panel image.

/// When the program loses focus, click outside the program's Game screen focus is false 
        public  void  OnApplicationFocus ( bool focus)
         {
             if (!focus)
            {
                //Get the pixel color of the center point on the image 
                Color = m_screenImage.sprite.texture.GetPixel(m_imageMesh.XAxisCount / 2 + 1 , m_imageMesh.YAxisCount / 2 + 1 );
                SetNoniusPositionByColor();
                WorkState = E_WorkState.Normal;
            }
        }

  1. Comment out the contents of the OnDisable function on the script.
    Because the color picker panel is initially hidden, this function will be called when it is hidden. If the button event listeners are removed, it will not be able to respond to mouse events, so you need to comment or delete.

2. Use the modified ColorPicker plug-in color picker function to modify the color of Image and Text

  1. Build a test scene
    Create a new image and text respectively to test the function of the color picker after the color picker is modified.
  2. Write a test script
    because the ColorPicker script provides us with color accessors on the color picker panel:
    So in the object to be modified and the color script of the UI, declare the ColorPicker object and call the color accessor of the object.
    As the following script looks like:

using UnityEngine;
using UnityEngine.UI;

public class SetColorTest : MonoBehaviour
{
    public GameObject colorPanel; //color picker panel 
    public Image setColorImg; //image that needs to be modified 
    public Text setColorText; //text that needs to be modified 
    public Button closeBtn;
     private Button pickBtn;
     private SpringGUI.ColorPicker picker; //Declare one Color picker object 
    private  bool bPickColor = false ; //whether to pick color 
    private  void  Start ()
    {
        pickBtn = setColorImg.GetComponent<Button>();
        picker = colorPanel.transform.Find("ColorPicker").GetComponent<SpringGUI.ColorPicker>();
        colorPanel.SetActive(false);
        closeBtn.onClick.AddListener(() =>
        {
            colorPanel.SetActive( false ); //Close the color picker 
            bPickColor = false ;
        });
        pickBtn.onClick.AddListener(() =>
        {
            colorPanel.gameObject.SetActive( true ); //Open the color picker 
            bPickColor = true ;
        });
    }
    private void Update()
    {
        if (bPickColor && picker!=null && picker.gameObject.activeSelf)
        {
            //Update the tested Image and Texty color to the color selected by the color picker panel
            setColorImg.color = picker.Color;
            setColorText.color = picker.Color;
        }
    }
}

3. Test the modified color picker function

1. In-editor testing
2. Test after packaging the exe

Perfect! perfect 😄

Leave a Reply

Your email address will not be published.