AForge.NET

  :: AForge.NET Framework :: Articles :: Forums ::

Comparing image to clone of itself gives unexpected results.

The forum is to discuss topics related to different areas of image processing and computer vision.

Comparing image to clone of itself gives unexpected results.

Postby 360 » Wed Jun 25, 2014 10:02 pm

Hi,

I'm building a character recognizing app.

Image

I wrote a CharacterMatcher class expressing a score on how identical two images are.

To check i created a unitTest to compare each character that i have to a clone of itself.
I would expect all characters to have a 100% match. But i got interesting results.

Image

Code: Select all
       [TestMethod]
        public void ScoreCharachterMatch_Show_mismatch_in_all_characters()
        {
            // Arrange
            var drawer = new CanvasDrawer();

            var characters = new Dictionary<char, Bitmap>
            {
                {'0',Characters._0},
                {'1',Characters._1},
                {'2',Characters._2},
                {'3',Characters._3},
                {'4',Characters._4},
                {'5',Characters._5},
                {'6',Characters._6},
                {'7',Characters._7},
                {'8',Characters._8},
                {'9',Characters._9},
                {'A',Characters.A},
                {'B',Characters.B},
                {'C',Characters.C},
                {'D',Characters.D},
                {'E',Characters.E},
                {'F',Characters.F},
            };

            var characterMatchers = characters.ToDictionary(x => x.Key, v => new CharacterMatcher(v.Value));

            // Act
            var images = characters.ToDictionary(k => k.Key, v => characterMatchers[v.Key].ScoreCharacterMatch<ResizeNearestNeighbor>(AForge.Imaging.Image.Clone(v.Value)))
                                   .ToDictionary(x => x.Key, x =>
                                   {
                                     var image = characters[x.Key];

                                     image = drawer.DrawPoints(image, characterMatchers[x.Key].MismatchInSource.Select(p => p.Key).ToList(), Color.Blue);
                                     image = drawer.DrawPoints(image, characterMatchers[x.Key].MismatchInCandidate.Select(p => p.Key).ToList(), Color.Yellow);
                                     return image;
                                   });

            // Assert
            characterMatchers.SelectMany(x => x.Value.MismatchInSource).Should().BeEmpty(String.Join("\n",
                       characterMatchers.Select(x => String.Format("{0}  {1} points mismatch", x.Key, x.Value.MismatchInSource.Count))));
        }



I iterate over all the pixels

Code: Select all
        private unsafe void IteratePixels(UnmanagedImage src, UnmanagedImage candidate, Action<PixelInfo> onPixel)
        {
            byte* ptrSrc = (byte*)src.ImageData.ToPointer();
            byte* ptrCandidate = (byte*)candidate.ImageData.ToPointer();

            int pixelSizeSrc = System.Drawing.Image.GetPixelFormatSize(src.PixelFormat);
            int pixelSizeCandidate = System.Drawing.Image.GetPixelFormatSize(src.PixelFormat);

            int offsetSrc = src.Stride - src.Width * pixelSizeSrc;
            int offsetCandidate = candidate.Stride - candidate.Width * pixelSizeCandidate;

            for (int y = 0; y < src.Height; y++)
            {
                for (int x = 0; x < src.Width; x++, ptrSrc += pixelSizeSrc, ptrCandidate += pixelSizeCandidate)
                {
                    var pixelInfo = new PixelInfo
                    {
                        X = x,
                        Y = y,
                        HasPixelSrc = HasPixel(ptrSrc),
                        HasPixelCandidate = HasPixel(ptrCandidate),
                        ColorSrc = GetColor(ptrSrc),
                        ColorCandidate = GetColor(ptrCandidate)
                    };

                    onPixel(pixelInfo);
                }
                ptrSrc += offsetSrc;
                ptrCandidate += offsetCandidate;
            }
        }

        private unsafe bool HasPixel(byte* ptr)
        {
            return !(ptr[RGB.R] == 0 && ptr[RGB.G] == 0 && ptr[RGB.B] == 0);
        }


The code compares every Point(x,y) in the source image to the candidate image, and checks if they match (either they have a color, or they don't).
After that a score is calculated ranging from 0..100 %

Code: Select all
        private void ScoreCharacter(PixelInfo pixel)
        {
            if (pixel.HasPixelSrc && (pixel.HasPixelSrc != pixel.HasPixelCandidate))
            {
                MismatchInSource.Add(new IntPoint(pixel.X, pixel.Y), pixel.ColorSrc);
            }
            else if (!pixel.HasPixelSrc && (pixel.HasPixelSrc != pixel.HasPixelCandidate))
            {
                MismatchInCandidate.Add(new IntPoint(pixel.X, pixel.Y), pixel.ColorCandidate);
            }

            ScoreCharacter(pixel.HasPixelSrc, pixel.HasPixelCandidate);
        }

        private void ScoreCharacter(bool srcHasPixel, bool candidateHasPixel)
        {
            if (srcHasPixel)
            {
                _nrOfPixelsInSource++;
            }
            if (srcHasPixel && candidateHasPixel)
            {
                _pixelMatchInCandidate++;
            }
        }


But in each character there is a small deviation.

blue: source image has colored pixel, but candidate picture hasn't
yellow: candidate image has colored pixel, but source picture hasn't


If there is reference equality on the pictures everything works.
The pictures are not rotated or resized because (naturally) they have the same format.

for clones i used
- Aforge.Imaging.Image.Clone
- new BItmap(image) // which produces even worse results.


What am i missing here? ;)
360
 
Posts: 1
Joined: Wed Jun 25, 2014 9:02 pm



Return to Image Processing and Computer Vision