fbpx

Working With Mouse & Trackbar Events in OpenCV | Creating Instagram Filters – Pt ⅓

By Taha Anwar and Rizwan Naeem

On January 31, 2022

Watch Video Here

You must have tried or heard of the famous Instagram filters, if you haven’t then … well 🤔 please just let me know the year you are living in, along with the address of your cave xD in the comments section, I would love to visit you (I mean visit the past) someday. These filters are everywhere nowadays, every social media person is obsessed with these.

Being a vison/ml practitioner, you must have thought about creating one or at least have wondered how these filters completely change the vibe of an image. If yes, then here at Bleed AI we have published just the right series for you (Yes you heard right a complete series), in which you will learn to create some fascinating photo filters along with a user interface similar to the Instagram filter selection screen using OpenCV in python.

In Instagram (or any other photo filter application), we touch on the screen to select different filters from a list of filters previews to apply them to an image, similarly, if you want to select a filter (using a mouse) and apply it to an image in python, you might want to use OpenCV, specifically OpenCV’s Mouse events, and these filter applications normally also provide a slider to adjust the intensity of the selected filter, we can create something similar in OpenCV using a trackbar.

So in this tutorial, we will cover all the nitty-gritty details required to use Mouse Events (to select a filter) and TrackBars (to control the intensity of filters) in OpenCV, and to kill the dryness we will learn all these concepts by building some mini-applications, so trust me you won’t get bored.

This is the first tutorial in our 3 part Creating Instagram Filters series. All three posts are titled as:

  • Part 1: Working With Mouse & Trackbar Events in OpenCV (Current tutorial)
  • Part 2: Working With Lookup Tables & Applying Color Filters on Images & Videos
  • Part 3: Designing Advanced Image Filters in OpenCV

Outline

This tutorial can be split into the following parts:

Alright, let’s get started.

Download Code:

Import the Libraries

First, we will import the required libraries.

Introduction to Mouse Events in OpenCV

Well, mouse events in OpenCV are the events that are triggered when a user interacts with an OpenCV image window using a mouse. OpenCV allows you to capture different types of mouse events like left-button down, left-button up, left-button double-click, etc, and then whenever these events occur, you can then execute some operation(s) accordingly, e.g. apply a certain filter.

Here are the most common mouse events that you can work with

Event IDEnumeratorEvent Indication
0cv2.EVENT_MOUSEMOVEIndicates that the mouse pointer has moved over the window.
1cv2.EVENT_LBUTTONDOWNIndicates that the left mouse button is pressed.
2cv2.EVENT_RBUTTONDOWNIndicates that the right mouse button is pressed.
3cv2.EVENT_MBUTTONDOWNIndicates that the middle mouse button is pressed.
4cv2.EVENT_LBUTTONUPIndicates that the left mouse button is released.
5cv2.EVENT_RBUTTONUPIndicates that the right mouse button is released.
6cv2.EVENT_MBUTTONUPIndicates that the middle mouse button is released.
7cv2.EVENT_LBUTTONDBLCLKIndicates that the left mouse button is double-clicked.
8cv2.EVENT_RBUTTONDBLCLKIndicates that the right mouse button is double-clicked.
9cv2.EVENT_MBUTTONDBLCLKIndicates that the middle mouse button is double-clicked.

I have only mentioned the most commonly triggered events with their Event IDs and Enumerators. You can check cv2.MouseEventTypes for the remainings.

Now for capturing these events, we will have to attach an event listener to an image window, so in simple words; we are just gonna be telling the OpenCV library to start reading the mouse input on an image window, this can be done easily by using the cv2.setMouseCallback() function.

Function Syntax:

cv2.setMouseCallback(winname, onMouse, userdata)

Parameters:

  • winname: – The name of the window with which we’re gonna attach the mouse event listener.
  • onMouse: – The method (callback function) that is going to be called every time a mouse event is captured.
  • userdata: (optional) – A parameter passed to the callback function.

Now before we could use the above function two things should be done, first we must create a window beforehand since we will have to pass the window name to the cv2.setMouseCallback() function. For this we will use the cv2.namedWindow(winname) function.

And the next thing we must do is to create a method (callback function) that is going to be called whenever a mouse event is captured. And this method by default will have a couple of arguments containing info related to the captured mouse event.

Creating a Paint Application utilizing Mouse Events

Now we will create a callback function drawShapes(), that will draw a circle or rectangle on an empty canvas (i.e. just an empty black image) at the location of the mouse cursor whenever the left or right mouse button is pressed respectively and clear the canvas whenever the middle mouse button is pressed.

Now it’s time to draw circles and rectangles on a webcam feed utilizing mouse events in real-time, as we have created a named window Webcam Feed and a callback function drawShapes() (to draw on a canvas), so we are all set to use the function cv2.setMouseCallback() to serve the purpose.

Output Video:

Working as expected! but there’s a minor issue, we can only draw fixed size shapes so let’s try to overcome this limitation by creating another callback function drawResizableShapes() that will use the cv2.EVENT_MOUSEMOVE event, to measure the required size of a shape in real-time meaning the user will have to drag the mouse while pressing the right or left mouse button to draw shapes of different sizes on the canvas.

Now we are all set to overcome that same size limitation, we will utilize this drawResizableShapes() callback function created above, to draw circles and rectangles of various sizes on a webcam feed utilizing mouse events.

Output Video:

Cool! right? feels like a mini paint application but still, something’s missing. How about adding a feature for users to paint (draw anything) with different colors to select from, and erase the drawings, on the webcam feed. All this just by utilizing mouse events in OpenCV, feels like a plan right? let’s create it. Again first we will have to create a callback function draw() that will carry all the heavy burden of drawing, erasing, and selecting paint color utilizing mouse events.

Now that we have created a drawing callback function draw(), it’s time to use it to create that paint application we had in mind, the application will draw, erase on a webcam feed with different colors utilizing mouse events in real-time.

Output Video:

Awesome! Everything went according to the plan, the application is working fine. But there’s a minor issue that we have limited options to choose the paint color from. We can add more colors previews on the frame and add code to select those colors using mouse events but that will take forever, I wish there was a simpler way.

Working with TrackBars in OpenCV

Well, there’s a way to get around this i.e., using TrackBars, as I mentioned at the beginning of the tutorial, these are like sliders with a minimum and a maximum value and allow users to slide across and select a value. These are extremely beneficial in adjusting the parameters of things in code in real-time instead of manually changing them and running the code again and again. For our case, these can be very handy to choose filters intensity and paint color (RGB) value in real-time.

OpenCV allows creating trackbars by using the cv2.createTrackbar() function. The procedure is pretty similar to that of cv2.setMouseCallback() function, first we will have to create a namedwindow, then create a method (i.e. called onChange in the slider) and finally attach the trackbar to that window using the function cv2.createTrackbar().

Function Syntax:

cv2.createTrackbar(trackbarname,winname,value,count,onChange)

Parameters:

  • trackbarname: It is the name of the created trackbar.
  • winname: It is the name of the window that will be attached to the created trackbar.
  • value: It is the starting value for the slider. When the program starts, this is the point where the slider will be at.
  • count It is the max value for the slider. The min value is always 0.
  • onChange: It is the method that is called whenever the position of the slider is changed.

And to get the value of the slider we will have to use another function cv2.getTrackbarPos().

Function Syntax:

cv2.getTrackbarPos(Trackbar_Name,winname)

Parameters:

  • Trackbar_Name: It is the name of the trackbar you wish to get the value of.
  • winname: It is the name of the window that the trackbar is attached to.

Now let’s create a simple python script that will utilize trackbars to move a circle around in a webcam feed window and adjust its radius in real-time.

Output Video:

I don’t know why, but this kind of reminds me of my childhood when I used to spend hours playing that famous Bouncing Ball Game on my father’s Nokia phone 😂. But the ball (circle) we moved using trackbars wasn’t bouncing, in fact there was no game mechanics, but hey you can actually change that if you want by adding actual physical properties ( like mass, force, acceleration, and everything ) to this ball (circle) using Pymunk library.

And I have made something similar in our latest course Computer Vision For Building Cutting Edge Applications too, by Combining Physics and Computer Vision, so do check that out, if you are interested in building complex, real-world, and thrilling AI applications.

Assignment (Optional)

Create 3 trackbars to control the RGB paint color in the paint application above and draw a resizable Ellipse on webcam feed utilizing mouse events and share the results with me in the comments section.

Additional Resources

Summary

In today’s tutorial, we went over almost all minor details regarding Mouse Events and TrackBars and used them to make a few fun applications. 

First, we used mouse events to draw fixed size shapes, then we realized this size limitation and got around it by drawing shapes of different sizes.  After that, we created a mini paint application capable of drawing anything, it had 3 different colors to select from and also had an option for erasing the drawings. And all of this ran on the live webcam feed. We then also learned about TrackBars in OpenCV and why they are useful and then we utilized them to move a resizable circle around on a webcam feed.

Also, don’t forget that our ultimate goal for creating all these mini-applications was to get you familiar with Mouse Events and TrackBars. As we will need these to select a filter and change the applied filter intensity in real-time in the next post of this series, so buckle up, as things are about to get more interesting in the next week’s post.

Let me know in the comments If you have any questions!

Designing Advanced Image Filters in OpenCV | Creating Instagram Filters – Pt 3⁄3

Designing Advanced Image Filters in OpenCV | Creating Instagram Filters – Pt 3⁄3

This is the last tutorial in our 3 part Creating Instagram Filters series. In this tutorial, you will learn to create 10 very interesting and cool Instagram filters-like effects on images and videos. The Filters which are gonna be covered are; Warm Filter, Cold Filter, Gotham Filter, GrayScale Filter, Sepia Filter, Pencil Sketch Filter, Sharpening Filter, Detail Enhancing Filter, Invert Filter, and Stylization Filter.

Working With Lookup Tables & Applying Color Filters on Images & Videos | Creating Instagram Filters – Pt ⅔

Working With Lookup Tables & Applying Color Filters on Images & Videos | Creating Instagram Filters – Pt ⅔

This is the second tutorial in our 3 part Creating Instagram Filters series. In this tutorial, you will learn what LookUp Tables are, why are they preferred along with their use cases in real-life, and then utilize these LookUp Tables to create some spectacular photo effects called Color Filters a.k.a. Tone Effects. And then you will create a user interface similar to the Instagram filter selection screen using mouse events & trackbars in OpenCV.

0 Comments

Submit a Comment

Your email address will not be published.