Output image diffs

[jeffcressman]

I was expecting that I would get an image diff as output when there is an error, similar to the image in the Readme.

Unless I am running the tests incorrectly I am not seeing this as output.

For a complex page it would be helpful to know where exactly on the page the problem is.

[DigitalProducts]

hi, can you provide your test script and your configuration? also try to run the debug mode, there you can see were the tool tries to store the image.

[sebastianneubert]

Hi.

I added a function to compare images of different sizes. IMagick would throw an exception, if you compare images which have no equal width and height parameters. So you could only see the screenshot of the failed codeception test.

Before comparison of the images, they will be normalized in their proportions. Now they produce the comparing image like expected.

[jeffcressman]

Hi.

I just checked out the repository again and ran the visualceptions tests. I
think there is still something wrong. The last time I ran the tests, before
your update, images were generated. Now, with the same setup, only blank
images are generated.

I'm running VisualCeption in Ubuntu 12.04 via Vagrant and VirtualBox and
using Phantomjs for WebDriver. I have no problem using Phantomjs or
Casperjs directly for screenshots.

I've attached my _log directory after running the bundled visualception
tests.

This is the output of my run:

vagrant@precise64:~/visualception/test/integration$ codecept run

Codeception PHP Testing Framework v1.8.0

Powered by PHPUnit 3.7.37 by Sebastian Bergmann.

*Acceptance Tests (6) *

Error

Error

Error

Error

Error

Running * (SimpleCept.php)*
Error

---

Time: 6.8 seconds, Memory: 17.00Mb

There were 6 errors:

---

1. Failed to see visual changes after size changes in
   NotSameSizeCest.seeVisualChangesAfterSizeChanges

#1
/home/vagrant/test-suite/vendor/facebook/webdriver/lib/remote/HttpCommandExecutor.php:232

#2
/home/vagrant/test-suite/vendor/facebook/webdriver/lib/remote/HttpCommandExecutor.php:133

#3
/home/vagrant/test-suite/vendor/facebook/webdriver/lib/remote/HttpCommandExecutor.php:103

#4
/home/vagrant/test-suite/vendor/facebook/webdriver/lib/remote/RemoteWebDriver.php:181

#5 /home/vagrant/visualception/module/VisualCeption.php:50

#6
/home/vagrant/test-suite/vendor/symfony/event-dispatcher/Symfony/Component/EventDispatcher/EventDispatcher.php:164

#7
/home/vagrant/test-suite/vendor/symfony/event-dispatcher/Symfony/Component/EventDispatcher/EventDispatcher.php:53

---

2. Failed to see visual changes in TimeComparisonCest.seeVisualChanges

#1
/home/vagrant/test-suite/vendor/facebook/webdriver/lib/remote/HttpCommandExecutor.php:232

#2
/home/vagrant/test-suite/vendor/facebook/webdriver/lib/remote/HttpCommandExecutor.php:133

#3
/home/vagrant/test-suite/vendor/facebook/webdriver/lib/remote/HttpCommandExecutor.php:103

#4
/home/vagrant/test-suite/vendor/facebook/webdriver/lib/remote/RemoteWebDriver.php:181

#5 /home/vagrant/visualception/module/VisualCeption.php:50

#6
/home/vagrant/test-suite/vendor/symfony/event-dispatcher/Symfony/Component/EventDispatcher/EventDispatcher.php:164

#7
/home/vagrant/test-suite/vendor/symfony/event-dispatcher/Symfony/Component/EventDispatcher/EventDispatcher.php:53

---

3. Failed to dont see visual changes in
   TimeComparisonCest.dontSeeVisualChanges

#1
/home/vagrant/test-suite/vendor/facebook/webdriver/lib/remote/HttpCommandExecutor.php:232

#2
/home/vagrant/test-suite/vendor/facebook/webdriver/lib/remote/HttpCommandExecutor.php:133

#3
/home/vagrant/test-suite/vendor/facebook/webdriver/lib/remote/HttpCommandExecutor.php:103

#4
/home/vagrant/test-suite/vendor/facebook/webdriver/lib/remote/RemoteWebDriver.php:181

#5 /home/vagrant/visualception/module/VisualCeption.php:50

#6
/home/vagrant/test-suite/vendor/symfony/event-dispatcher/Symfony/Component/EventDispatcher/EventDispatcher.php:164

#7
/home/vagrant/test-suite/vendor/symfony/event-dispatcher/Symfony/Component/EventDispatcher/EventDispatcher.php:53

---

4. Failed to see visual changes and hide element in
   TimeComparisonCest.seeVisualChangesAndHideElement

#1
/home/vagrant/test-suite/vendor/facebook/webdriver/lib/remote/HttpCommandExecutor.php:232

#2
/home/vagrant/test-suite/vendor/facebook/webdriver/lib/remote/HttpCommandExecutor.php:133

#3
/home/vagrant/test-suite/vendor/facebook/webdriver/lib/remote/HttpCommandExecutor.php:103

#4
/home/vagrant/test-suite/vendor/facebook/webdriver/lib/remote/RemoteWebDriver.php:181

#5 /home/vagrant/visualception/module/VisualCeption.php:50

#6
/home/vagrant/test-suite/vendor/symfony/event-dispatcher/Symfony/Component/EventDispatcher/EventDispatcher.php:164

#7
/home/vagrant/test-suite/vendor/symfony/event-dispatcher/Symfony/Component/EventDispatcher/EventDispatcher.php:53

---

5. Failed to dont see visual changes and hide element in
   TimeComparisonCest.dontSeeVisualChangesAndHideElement

#1
/home/vagrant/test-suite/vendor/facebook/webdriver/lib/remote/HttpCommandExecutor.php:232

#2
/home/vagrant/test-suite/vendor/facebook/webdriver/lib/remote/HttpCommandExecutor.php:133

#3
/home/vagrant/test-suite/vendor/facebook/webdriver/lib/remote/HttpCommandExecutor.php:103

#4
/home/vagrant/test-suite/vendor/facebook/webdriver/lib/remote/RemoteWebDriver.php:181

#5 /home/vagrant/visualception/module/VisualCeption.php:50

#6
/home/vagrant/test-suite/vendor/symfony/event-dispatcher/Symfony/Component/EventDispatcher/EventDispatcher.php:164

#7
/home/vagrant/test-suite/vendor/symfony/event-dispatcher/Symfony/Component/EventDispatcher/EventDispatcher.php:53

---

6. SimpleCept.php

#1
/home/vagrant/test-suite/vendor/facebook/webdriver/lib/remote/HttpCommandExecutor.php:232

#2
/home/vagrant/test-suite/vendor/facebook/webdriver/lib/remote/HttpCommandExecutor.php:133

#3
/home/vagrant/test-suite/vendor/facebook/webdriver/lib/remote/HttpCommandExecutor.php:103

#4
/home/vagrant/test-suite/vendor/facebook/webdriver/lib/remote/RemoteWebDriver.php:181

#5 /home/vagrant/visualception/module/VisualCeption.php:50

#6
/home/vagrant/test-suite/vendor/symfony/event-dispatcher/Symfony/Component/EventDispatcher/EventDispatcher.php:164

#7
/home/vagrant/test-suite/vendor/symfony/event-dispatcher/Symfony/Component/EventDispatcher/EventDispatcher.php:53

FAILURES!

Tests: 6, Assertions: 0, Errors: 6.

vagrant@precise64:~/visualception/test/integration$

Also, before running the tests I always have to create the _log directory.
You may want to add that to the repository.

Cheers,

Jeff

On Thu, May 22, 2014 at 10:02 PM, Sebastian Neubert <
[email protected]> wrote:

Hi.

I added a function to compare images of different sizes. IMagick would
throw an exception, if you compare images which have no equal width and
height parameters. So you could only see the screenshot of the failed
codeception test.

Before comparison of the images, they will be normalized in their
proportions. Now they produce the comparing image like expected.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/31#issuecomment-43881647
.

[sebastianneubert]

Oh, ok. It looks like PhantomJS webdriver could not inject the JQuery Javascript framework. In your logs appears line 50 of VisualCeption.php

        $this->webDriver->executeScript($jQueryString);

We used the facebook webdriver to connect with selenium and firefox.

First of all, please update codeception. 1.8.0 is a little bit older and there were some nice updates, including webdriver features. 1.8.5 is stable and works fine.

Next step, run your tests with the debug option -d. So you can see a little bit more than "error".

$ codecept run -d

If there is really a difference between phantom webdriver and facebook webdriver, there is a problem. Because of the way Visualception works it needs to inject JQuery. In this case: if you are familar with phantom and casper, then try PhantomCSS https://github.com/Huddle/PhantomCSS

Maybe it's the better visual regression tool for you.

HTH
Kind regards
Sebastian