Discussion:
Extending TDD to ATDD
Conrad Ononeme
2011-02-26 19:30:20 UTC
Permalink
Hello ALL!

I have a quick question.
I am the lead embedded tester in my team that has done very well in the uptake of Agile.

Now having successfully gotten the team to follow best practice in pretty much every other area of Agile Software development, I now want to push the boat out a little by seeing if we can actually extend the developers' tests to form the basis (or at least part of it) of our Acceptance Tests.
Currently the practice is that we refine the Acceptance Criteria once a story is in play, as part of our iteration activities, and that process in turn serves out or Acceptance tests.
The technical tester then automates the tests (identified as automation candidates) while the alternative flows and negative testing scenarios are manually run and validated as the main part of the Acceptance Tests.
The tester writes all the tests in Selenium DE only and at the moment too as far as I am aware the devs do too.

The devs as stated earlier, currently practice TDD.

My challenge is how to make better use of their assets by finding a way to extend their tests to at least form part of the basis of our Acceptance Tests.

I suspect however that selenium may have some limitations that may not make this ambition possible though. So in that case can someone advise what the best OS tools out there that can help will be? (preferably something light on tecchy savviness as I am not a `technical' tester at all :-)

I hope I haven't put all to sleep yet with my long message, but I thought as much background as possible would be germane at this point, so kindly do forgive me.

On that note u shall look forward to hearing your always illuminating ideas and experiences!

Many thanks!

CO
Sent from my Blackberry Device

------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/agile-testing/

<*> Your email settings:
Individual Email | Traditional

<*> To change settings online go to:
http://groups.yahoo.com/group/agile-testing/join
(Yahoo! ID required)

<*> To change settings via email:
agile-testing-digest-***@public.gmane.org
agile-testing-fullfeatured-***@public.gmane.org

<*> To unsubscribe from this group, send an email to:
agile-testing-unsubscribe-***@public.gmane.org

<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
Markus Gaertner
2011-02-27 12:09:09 UTC
Permalink
Hi Conrad,

TDD and ATDD have different goals. With TDD you drive the design of the
implementation while ATDD drives the requirements of the system. So, I wouldn't
want to mix the two in first place, and an overlap from both perspectives is
indeed fine.

However, there might be occasions where driving the design of your system based
upon the knowledge you gained from your acceptance test can be worthwhile. The
idea is to automate the acceptance test, and taking that knowledge to inform
the design decisions for your classes. I am currently thinking this through in
some more detail, but the idea is to automate your examples from the acceptance
criteria, and then starting over to develop the classes that will drive your
business model based upon that - and using TDD.

Best
Markus
Post by Conrad Ononeme
Hello ALL!
I have a quick question.
I am the lead embedded tester in my team that has done very well in the uptake of Agile.
Now having successfully gotten the team to follow best practice in pretty much every other area of Agile Software development, I now want to push the boat out a little by seeing if we can actually extend the developers' tests to form the basis (or at least part of it) of our Acceptance Tests.
Currently the practice is that we refine the Acceptance Criteria once a story is in play, as part of our iteration activities, and that process in turn serves out or Acceptance tests.
The technical tester then automates the tests (identified as automation candidates) while the alternative flows and negative testing scenarios are manually run and validated as the main part of the Acceptance Tests.
The tester writes all the tests in Selenium DE only and at the moment too as far as I am aware the devs do too.
The devs as stated earlier, currently practice TDD.
My challenge is how to make better use of their assets by finding a way to extend their tests to at least form part of the basis of our Acceptance Tests.
I suspect however that selenium may have some limitations that may not make this ambition possible though. So in that case can someone advise what the best OS tools out there that can help will be? (preferably something light on tecchy savviness as I am not a `technical' tester at all :-)
I hope I haven't put all to sleep yet with my long message, but I thought as much background as possible would be germane at this point, so kindly do forgive me.
On that note u shall look forward to hearing your always illuminating ideas and experiences!
Many thanks!
CO
Sent from my Blackberry Device
------------------------------------
Yahoo! Groups Links
--
Markus Gaertner
http://blog.shino.de
Twitter: @mgaertne


------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/agile-testing/

<*> Your email settings:
Individual Email | Traditional

<*> To change settings online go to:
http://groups.yahoo.com/group/agile-testing/join
(Yahoo! ID required)

<*> To change settings via email:
agile-testing-digest-***@public.gmane.org
agile-testing-fullfeatured-***@public.gmane.org

<*> To unsubscribe from this group, send an email to:
agile-testing-unsubscribe-***@public.gmane.org

<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
Franz Allan Valencia See
2011-02-27 13:42:22 UTC
Permalink
Good day,

I would suggest the other way around - use ATDD to drive TDD. Your technical
testers can create the specifications (automated acceptance tests written in
a specification manner instead of a script-style manner), while your devs
would try to make those specifications pass by writing the code in
TDD-style.

This would mean that the specifications that your technical testers will
write would have to be shared to your developers. These specifications may
be unimplemented at start (i.e. for those written in Selenium IDE, it can be
via from the name of the test, or from the comments on the test ), and then
this should be shared to the developers so that the devs would TDD their way
to make that specification pass. On parallel, your technical testers can
start implementing these specifications. Thus, once the devs are finished,
your technical testers (or your devs) can run the implemented specifications
to see whether the feature / user story has really be implemented. ...Or if
there are too many specifications to be implemented, then your devs can work
on those as well.

However, if you want to have more powerful assertions that currently
Selenium IDE does not support out of the box, then you export the scripts in
your Selenium IDE in a programming language that your team is comfortable
with. This would then allow you to use your developers to write custom
assertions that your technical testers can use.

Cheers,
--
Franz Allan Valencia See | Java Software Engineer
franz.see-***@public.gmane.org
LinkedIn: http://www.linkedin.com/in/franzsee
Twitter: http://www.twitter.com/franz_see
Post by Markus Gaertner
Hi Conrad,
TDD and ATDD have different goals. With TDD you drive the design of the
implementation while ATDD drives the requirements of the system. So, I wouldn't
want to mix the two in first place, and an overlap from both perspectives is
indeed fine.
However, there might be occasions where driving the design of your system based
upon the knowledge you gained from your acceptance test can be worthwhile. The
idea is to automate the acceptance test, and taking that knowledge to inform
the design decisions for your classes. I am currently thinking this through in
some more detail, but the idea is to automate your examples from the acceptance
criteria, and then starting over to develop the classes that will drive your
business model based upon that - and using TDD.
Best
Markus
Post by Conrad Ononeme
Hello ALL!
I have a quick question.
I am the lead embedded tester in my team that has done very well in the
uptake of Agile.
Post by Conrad Ononeme
Now having successfully gotten the team to follow best practice in pretty
much every other area of Agile Software development, I now want to push the
boat out a little by seeing if we can actually extend the developers' tests
to form the basis (or at least part of it) of our Acceptance Tests.
Post by Conrad Ononeme
Currently the practice is that we refine the Acceptance Criteria once a
story is in play, as part of our iteration activities, and that process in
turn serves out or Acceptance tests.
Post by Conrad Ononeme
The technical tester then automates the tests (identified as automation
candidates) while the alternative flows and negative testing scenarios are
manually run and validated as the main part of the Acceptance Tests.
Post by Conrad Ononeme
The tester writes all the tests in Selenium DE only and at the moment too
as far as I am aware the devs do too.
Post by Conrad Ononeme
The devs as stated earlier, currently practice TDD.
My challenge is how to make better use of their assets by finding a way
to extend their tests to at least form part of the basis of our Acceptance
Tests.
Post by Conrad Ononeme
I suspect however that selenium may have some limitations that may not
make this ambition possible though. So in that case can someone advise what
the best OS tools out there that can help will be? (preferably something
light on tecchy savviness as I am not a `technical' tester at all :-)
Post by Conrad Ononeme
I hope I haven't put all to sleep yet with my long message, but I thought
as much background as possible would be germane at this point, so kindly do
forgive me.
Post by Conrad Ononeme
On that note u shall look forward to hearing your always illuminating
ideas and experiences!
Post by Conrad Ononeme
Many thanks!
CO
Sent from my Blackberry Device
------------------------------------
Yahoo! Groups Links
--
Markus Gaertner
http://blog.shino.de
pepijnvandevorst
2011-02-27 19:05:16 UTC
Permalink
Hello Conrad,

Did you consider FitNesse?
From your post it's not clear to me why you think Selenium isn't good enough to use for ATDD. But FitNesse is often used to automate acceptance tests.
Regards,
Pepijn
Hello ALL!
I have a quick question.
I am the lead embedded tester in my team that has done very well in the uptake of Agile.
Now having successfully gotten the team to follow best practice in pretty much every other area of Agile Software development, I now want to push the boat out a little by seeing if we can actually extend the developers' tests to form the basis (or at least part of it) of our Acceptance Tests.
Currently the practice is that we refine the Acceptance Criteria once a story is in play, as part of our iteration activities, and that process in turn serves out or Acceptance tests.
The technical tester then automates the tests (identified as automation candidates) while the alternative flows and negative testing scenarios are manually run and validated as the main part of the Acceptance Tests.
The tester writes all the tests in Selenium DE only and at the moment too as far as I am aware the devs do too.
The devs as stated earlier, currently practice TDD.
My challenge is how to make better use of their assets by finding a way to extend their tests to at least form part of the basis of our Acceptance Tests.
I suspect however that selenium may have some limitations that may not make this ambition possible though. So in that case can someone advise what the best OS tools out there that can help will be? (preferably something light on tecchy savviness as I am not a `technical' tester at all :-)
I hope I haven't put all to sleep yet with my long message, but I thought as much background as possible would be germane at this point, so kindly do forgive me.
On that note u shall look forward to hearing your always illuminating ideas and experiences!
Many thanks!
CO
Sent from my Blackberry Device
------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/agile-testing/

<*> Your email settings:
Individual Email | Traditional

<*> To change settings online go to:
http://groups.yahoo.com/group/agile-testing/join
(Yahoo! ID required)

<*> To change settings via email:
agile-testing-digest-***@public.gmane.org
agile-testing-fullfeatured-***@public.gmane.org

<*> To unsubscribe from this group, send an email to:
agile-testing-unsubscribe-***@public.gmane.org

<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
Conrad Ononeme
2011-02-28 05:46:32 UTC
Permalink
Thank-You everyone for your insightful input as usual.
Fitness is definitely an option to consider....
Sent from my Blackberry Device

-----Original Message-----
From: "pepijnvandevorst" <***@gmail.com>
Sender: agile-***@yahoogroups.com
Date: Sun, 27 Feb 2011 19:05:16
To: <agile-***@yahoogroups.com>
Reply-To: agile-***@yahoogroups.com
Subject: [agile-testing] Re: Extending TDD to ATDD

Hello Conrad,

Did you consider FitNesse?

From your post it's not clear to me why you think Selenium isn't good enough to use for ATDD. But FitNesse is often used to automate acceptance tests.

Regards,
Pepijn
Post by Conrad Ononeme
Hello ALL!
I have a quick question.
I am the lead embedded tester in my team that has done very well in the uptake of Agile.
Now having successfully gotten the team to follow best practice in pretty much every other area of Agile Software development, I now want to push the boat out a little by seeing if we can actually extend the developers' tests to form the basis (or at least part of it) of our Acceptance Tests.
Currently the practice is that we refine the Acceptance Criteria once a story is in play, as part of our iteration activities, and that process in turn serves out or Acceptance tests.
The technical tester then automates the tests (identified as automation candidates) while the alternative flows and negative testing scenarios are manually run and validated as the main part of the Acceptance Tests.
The tester writes all the tests in Selenium DE only and at the moment too as far as I am aware the devs do too.
The devs as stated earlier, currently practice TDD.
My challenge is how to make better use of their assets by finding a way to extend their tests to at least form part of the basis of our Acceptance Tests.
I suspect however that selenium may have some limitations that may not make this ambition possible though. So in that case can someone advise what the best OS tools out there that can help will be? (preferably something light on tecchy savviness as I am not a `technical' tester at all :-)
I hope I haven't put all to sleep yet with my long message, but I thought as much background as possible would be germane at this point, so kindly do forgive me.
On that note u shall look forward to hearing your always illuminating ideas and experiences!
Many thanks!
CO
Sent from my Blackberry Device
Franz Allan Valencia See
2011-02-28 09:41:34 UTC
Permalink
Fitness is the defacto, but I would suggest you guys take a look at
Robotframework as well :-)

Our testers found it easier to use than FitNess. One of the major reasons
is because it has an auto-complete feature, which means testers do not have
to memorize the list of commands/keywords available.

Cheers,
--
Franz Allan Valencia See | Java Software Engineer
franz.see-***@public.gmane.org
LinkedIn: http://www.linkedin.com/in/franzsee
Twitter: http://www.twitter.com/franz_see
Post by Conrad Ononeme
Thank-You everyone for your insightful input as usual.
Fitness is definitely an option to consider....
Sent from my Blackberry Device
------------------------------
*Date: *Sun, 27 Feb 2011 19:05:16 -0000
*Subject: *[agile-testing] Re: Extending TDD to ATDD
Hello Conrad,
Did you consider FitNesse?
From your post it's not clear to me why you think Selenium isn't good
enough to use for ATDD. But FitNesse is often used to automate acceptance
tests.
Regards,
Pepijn
Post by Conrad Ononeme
Hello ALL!
I have a quick question.
I am the lead embedded tester in my team that has done very well in the
uptake of Agile.
Post by Conrad Ononeme
Now having successfully gotten the team to follow best practice in pretty
much every other area of Agile Software development, I now want to push the
boat out a little by seeing if we can actually extend the developers' tests
to form the basis (or at least part of it) of our Acceptance Tests.
Post by Conrad Ononeme
Currently the practice is that we refine the Acceptance Criteria once a
story is in play, as part of our iteration activities, and that process in
turn serves out or Acceptance tests.
Post by Conrad Ononeme
The technical tester then automates the tests (identified as automation
candidates) while the alternative flows and negative testing scenarios are
manually run and validated as the main part of the Acceptance Tests.
Post by Conrad Ononeme
The tester writes all the tests in Selenium DE only and at the moment too
as far as I am aware the devs do too.
Post by Conrad Ononeme
The devs as stated earlier, currently practice TDD.
My challenge is how to make better use of their assets by finding a way
to extend their tests to at least form part of the basis of our Acceptance
Tests.
Post by Conrad Ononeme
I suspect however that selenium may have some limitations that may not
make this ambition possible though. So in that case can someone advise what
the best OS tools out there that can help will be? (preferably something
light on tecchy savviness as I am not a `technical' tester at all :-)
Post by Conrad Ononeme
I hope I haven't put all to sleep yet with my long message, but I thought
as much background as possible would be germane at this point, so kindly do
forgive me.
Post by Conrad Ononeme
On that note u shall look forward to hearing your always illuminating
ideas and experiences!
Post by Conrad Ononeme
Many thanks!
CO
Sent from my Blackberry Device
Rob Park
2011-03-01 03:07:52 UTC
Permalink
I wouldn't necessarily assume FitNesse as a defacto.
As a former FitNesse user, I've been loving ATDD with Cucumber.
I've been using Cucumber with Celerity & Firewatir primarily, but many are
using Cucumber+Capybara to ATDD with Selenium.
I've heard good things about Concordian, Robot, and I think Lisa is still a
fan of Canoo, so there are many choices.
--
Rob
--
http://agileintention.blogspot.com
http://twitter.com/robpark
http://leandog.com



On Mon, Feb 28, 2011 at 2:41 AM, Franz Allan Valencia See <
Post by Franz Allan Valencia See
Fitness is the defacto, but I would suggest you guys take a look at
Robotframework as well :-)
Our testers found it easier to use than FitNess. One of the major reasons
is because it has an auto-complete feature, which means testers do not have
to memorize the list of commands/keywords available.
Cheers,
--
Franz Allan Valencia See | Java Software Engineer
LinkedIn: http://www.linkedin.com/in/franzsee
Twitter: http://www.twitter.com/franz_see
Post by Conrad Ononeme
Thank-You everyone for your insightful input as usual.
Fitness is definitely an option to consider....
Sent from my Blackberry Device
------------------------------
*Date: *Sun, 27 Feb 2011 19:05:16 -0000
*Subject: *[agile-testing] Re: Extending TDD to ATDD
Hello Conrad,
Did you consider FitNesse?
From your post it's not clear to me why you think Selenium isn't good
enough to use for ATDD. But FitNesse is often used to automate acceptance
tests.
Regards,
Pepijn
Post by Conrad Ononeme
Hello ALL!
I have a quick question.
I am the lead embedded tester in my team that has done very well in the
uptake of Agile.
Post by Conrad Ononeme
Now having successfully gotten the team to follow best practice in
pretty much every other area of Agile Software development, I now want to
push the boat out a little by seeing if we can actually extend the
developers' tests to form the basis (or at least part of it) of our
Acceptance Tests.
Post by Conrad Ononeme
Currently the practice is that we refine the Acceptance Criteria once a
story is in play, as part of our iteration activities, and that process in
turn serves out or Acceptance tests.
Post by Conrad Ononeme
The technical tester then automates the tests (identified as automation
candidates) while the alternative flows and negative testing scenarios are
manually run and validated as the main part of the Acceptance Tests.
Post by Conrad Ononeme
The tester writes all the tests in Selenium DE only and at the moment
too as far as I am aware the devs do too.
Post by Conrad Ononeme
The devs as stated earlier, currently practice TDD.
My challenge is how to make better use of their assets by finding a way
to extend their tests to at least form part of the basis of our Acceptance
Tests.
Post by Conrad Ononeme
I suspect however that selenium may have some limitations that may not
make this ambition possible though. So in that case can someone advise what
the best OS tools out there that can help will be? (preferably something
light on tecchy savviness as I am not a `technical' tester at all :-)
Post by Conrad Ononeme
I hope I haven't put all to sleep yet with my long message, but I
thought as much background as possible would be germane at this point, so
kindly do forgive me.
Post by Conrad Ononeme
On that note u shall look forward to hearing your always illuminating
ideas and experiences!
Post by Conrad Ononeme
Many thanks!
CO
Sent from my Blackberry Device
Markus Gaertner
2011-03-02 07:30:32 UTC
Permalink
In case you're looking for differences, you might want to take a look at the Triangle Test Sampler from Elisabeth Hendrickson. It solves the triangle tests on a web page using different frameworks. My fork is a bit more ahead right now also containing concordion tests:
https://github.com/MarkusGaertner/Triangle-Test-Sampler

Personally, I like using Cucumber similarly as Robot Framework (though I don't have an auto completion in my text editor of choice, which is vim or emacs), but I also love to do things with FitNesse. There are situations when I prefer one over the other, but that's rarely an impediment at all. In fact, I solved one problem in all of the three mentioned ones - each time a little bit different. All three were fun to deal with.

Best
Markus Gärtner
http://www.shino.de/blog
http://www.it-agile.de
@mgaertne
Post by Rob Park
I wouldn't necessarily assume FitNesse as a defacto.
As a former FitNesse user, I've been loving ATDD with Cucumber.
I've been using Cucumber with Celerity & Firewatir primarily, but many are
using Cucumber+Capybara to ATDD with Selenium.
I've heard good things about Concordian, Robot, and I think Lisa is still a
fan of Canoo, so there are many choices.
--
Rob
--
http://agileintention.blogspot.com
http://twitter.com/robpark
http://leandog.com
On Mon, Feb 28, 2011 at 2:41 AM, Franz Allan Valencia See <
Post by Franz Allan Valencia See
Fitness is the defacto, but I would suggest you guys take a look at
Robotframework as well :-)
Our testers found it easier to use than FitNess. One of the major reasons
is because it has an auto-complete feature, which means testers do not have
to memorize the list of commands/keywords available.
Cheers,
--
Franz Allan Valencia See | Java Software Engineer
LinkedIn: http://www.linkedin.com/in/franzsee
Twitter: http://www.twitter.com/franz_see
Post by Conrad Ononeme
Thank-You everyone for your insightful input as usual.
Fitness is definitely an option to consider....
Sent from my Blackberry Device
------------------------------
*Date: *Sun, 27 Feb 2011 19:05:16 -0000
*Subject: *[agile-testing] Re: Extending TDD to ATDD
Hello Conrad,
Did you consider FitNesse?
From your post it's not clear to me why you think Selenium isn't good
enough to use for ATDD. But FitNesse is often used to automate acceptance
tests.
Regards,
Pepijn
Post by Conrad Ononeme
Hello ALL!
I have a quick question.
I am the lead embedded tester in my team that has done very well in the
uptake of Agile.
Post by Conrad Ononeme
Now having successfully gotten the team to follow best practice in
pretty much every other area of Agile Software development, I now want to
push the boat out a little by seeing if we can actually extend the
developers' tests to form the basis (or at least part of it) of our
Acceptance Tests.
Post by Conrad Ononeme
Currently the practice is that we refine the Acceptance Criteria once a
story is in play, as part of our iteration activities, and that process in
turn serves out or Acceptance tests.
Post by Conrad Ononeme
The technical tester then automates the tests (identified as automation
candidates) while the alternative flows and negative testing scenarios are
manually run and validated as the main part of the Acceptance Tests.
Post by Conrad Ononeme
The tester writes all the tests in Selenium DE only and at the moment
too as far as I am aware the devs do too.
Post by Conrad Ononeme
The devs as stated earlier, currently practice TDD.
My challenge is how to make better use of their assets by finding a way
to extend their tests to at least form part of the basis of our Acceptance
Tests.
Post by Conrad Ononeme
I suspect however that selenium may have some limitations that may not
make this ambition possible though. So in that case can someone advise what
the best OS tools out there that can help will be? (preferably something
light on tecchy savviness as I am not a `technical' tester at all :-)
Post by Conrad Ononeme
I hope I haven't put all to sleep yet with my long message, but I
thought as much background as possible would be germane at this point, so
kindly do forgive me.
Post by Conrad Ononeme
On that note u shall look forward to hearing your always illuminating
ideas and experiences!
Post by Conrad Ononeme
Many thanks!
CO
Sent from my Blackberry Device
------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/agile-testing/

<*> Your email settings:
Individual Email | Traditional

<*> To change settings online go to:
http://groups.yahoo.com/group/agile-testing/join
(Yahoo! ID required)

<*> To change settings via email:
agile-testing-digest-***@public.gmane.org
agile-testing-fullfeatured-***@public.gmane.org

<*> To unsubscribe from this group, send an email to:
agile-testing-unsubscribe-***@public.gmane.org

<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
John Goodsen
2011-02-28 11:16:23 UTC
Permalink
We call it Outside-in development in the Ruby world, and we use Cucumber for
the Acceptance Tests (ATDD) and RSpec for the unit tests (TDD).

Here's one of many blogs that describe the process
http://rubylearning.com/blog/2010/10/05/outside-in-development/
--
John Goodsen RADSoft / Better Software Faster
jgoodsen-***@public.gmane.org Lean/Kanban/XP/Scrum Coaching and Training
http://www.radsoft.com Enterprise Ruby, Java and Scala Solutions
c***@public.gmane.org
2011-02-28 11:31:27 UTC
Permalink
Thanks John!
I will check it out now and revert with any questions I may have.


-----Original Message-----
From: John Goodsen <jgoodsen-***@public.gmane.org>
To: agile-testing-***@public.gmane.org
Sent: Mon, Feb 28, 2011 12:16 pm
Subject: Re: [agile-testing] Re: Extending TDD to ATDD





We call it Outside-in development in the Ruby world, and we use
Cucumber for the Acceptance Tests (ATDD) and RSpec for the unit tests
(TDD).

Here's one of many blogs that describe the process
http://rubylearning.com/blog/2010/10/05/outside-in-development/

--
John Goodsen                 RADSoft / Better Software Faster
jgoodsen-***@public.gmane.org            Lean/Kanban/XP/Scrum Coaching and
Training
http://www.radsoft.com          Enterprise Ruby, Java and Scala
Solutions








------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/agile-testing/

<*> Your email settings:
Individual Email | Traditional

<*> To change settings online go to:
http://groups.yahoo.com/group/agile-testing/join
(Yahoo! ID required)

<*> To change settings via email:
agile-testing-digest-***@public.gmane.org
agile-testing-fullfeatured-***@public.gmane.org

<*> To unsubscribe from this group, send an email to:
agile-testing-unsubscribe-***@public.gmane.org

<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
Mohinder
2011-02-28 10:14:00 UTC
Permalink
Hello Conrad,
I second Franz idea but stress that you should consider
other specification frameworks such as Cucumber and Gherkin before taking a
decision and see which fits your requirements best. I am saying from experience
that if you are to use Gojko Adzic approach of Specification By example and
building a live documentation from acceptance tests then you need a tool that
is flexible enough to add user tags of your choice. Also if you are planning to
automate your tests and going to use continuous integration via Hudson/Jenkins or
other tool then you need to consider that in your evaluation. Also to keep in
mind is how comfortable your developers are with using such tools. Robot
Framework and Cucumber would be my first choices but don’t take my word for it,
there are more equally good frameworks to choose from than the one listed here.
If you want to read on how to set up Cucumber then see the guide that Gojko has
written (See Link below). As far I know after talking to some people working on
Agile projects, Fitnesse is not the common choice for them due to its
limitation but still it is widely used.


Cucumber Guide: http://bit.ly/fKHjVN
With kind regards,



Mohinder Khosla, PhD
Adrian Stokes
2011-02-28 11:01:15 UTC
Permalink
Hi Conrad.

I would suggest reading Bridging the Communication Gap and Specification
by Example by Gojko Adzic. Details on Gojko.net. He talks of using
automated acceptance tests as living documentation. As long as you
automate literally your acceptance test is still an acceptance test even
when automated. We are working towards using FitNesse as our living
documentation by writing our automated tests as acceptance tests. We
have tried user stories and given, when, then and the wiki functionality
of FitNesse makes that possible. I can't comment on either Selenium or
Robotframework but I'm sure they all lend themselves to this practice.

I too am not a technical tester but am learning to use FitNesse. The
good thing about the wiki is I can write up my acceptance criteria,
examples of those that can become tests and those with the skills can
apply the automation. If the set up and tear down is already defined
you can create your own tests without that initial need for technical
skills.



Good luck. Ady



Ady Stokes
BI QA Analyst
Business Intelligence
BISD
Gateway House, Skipton
UK
Ext: 2452
Direct Dial: 0844 892 2596
www.hml.co.uk

---------------------------------------
The ability in financial outsourcing

From: agile-testing-***@public.gmane.org
[mailto:agile-testing-***@public.gmane.org] On Behalf Of Conrad Ononeme
Sent: 26 February 2011 19:30
To: agile-testing-***@public.gmane.org
Subject: [agile-testing] Extending TDD to ATDD





Hello ALL!

I have a quick question.
I am the lead embedded tester in my team that has done very well in the
uptake of Agile.

Now having successfully gotten the team to follow best practice in
pretty much every other area of Agile Software development, I now want
to push the boat out a little by seeing if we can actually extend the
developers' tests to form the basis (or at least part of it) of our
Acceptance Tests.
Currently the practice is that we refine the Acceptance Criteria once a
story is in play, as part of our iteration activities, and that process
in turn serves out or Acceptance tests.
The technical tester then automates the tests (identified as automation
candidates) while the alternative flows and negative testing scenarios
are manually run and validated as the main part of the Acceptance Tests.

The tester writes all the tests in Selenium DE only and at the moment
too as far as I am aware the devs do too.

The devs as stated earlier, currently practice TDD.

My challenge is how to make better use of their assets by finding a way
to extend their tests to at least form part of the basis of our
Acceptance Tests.

I suspect however that selenium may have some limitations that may not
make this ambition possible though. So in that case can someone advise
what the best OS tools out there that can help will be? (preferably
something light on tecchy savviness as I am not a `technical' tester at
all :-)

I hope I haven't put all to sleep yet with my long message, but I
thought as much background as possible would be germane at this point,
so kindly do forgive me.

On that note u shall look forward to hearing your always illuminating
ideas and experiences!

Many thanks!

CO
Sent from my Blackberry Device




*******************************************************************
This email is intended only for the addressee named above. As this email may contain confidential or privileged information, if you are not the named addressee or receive this message in error, please notify us immediately, delete it and do not make use of or copy it.

This message is protected by copyright. HML accepts no responsibility for viruses found in this message or any file attachment.

Homeloan Management Limited
Registered in England No. 2214839
The Bailey, Skipton, North Yorkshire, BD23 1DN

********************************************************************
charles_bradley_scrum_coach
2011-02-28 20:29:30 UTC
Permalink
Conrad,

You're on the right path, so keep up the good work.

First, if you're not already familiar with the "Testing Pyramid" concept, you need to be:
http://blog.mountaingoatsoftware.com/the-forgotten-layer-of-the-test-automation-pyramid
Also read the comments specifically the one about the "Cloud" that resides above the pyramid of "Exploratory Testing"

My advice:
Everyone so far has given good advice, of which the theme, I believe, is this: "Obtain and use a Service Layer testing tool like Cucumber, Fitnesse, etc to support your GUI testing(Sellenium)"

My main justification for this, is the same as Cohn's -- once you get over the learning curve with your service layer tool, tests to exercise the service layer (where a huge percentage of business logic resides... or SHOULD reside 'if your team is architecting/designing using best practices') are much cheaper and easier to maintain than GUI/Sellenium tests.

Another suggestion: I've found that, when I coach teams on heading down the "let's merge the testers' and developers' roads" like you are, an interesting phenomenon happens. Often times, a particular piece of logic will be well tested at the integration or unit level(which may or may not be exposed to your service testing tool). When developers and testers communicate about this, testers will find less need to test that logic at the GUI level(which is essentially also what happens when you do service layer testing -- less need to do GUI level testing) They'll still test, but they'll usually just that logic in an exploratory way(just a few boundary cases, rather than exhaustive testing) rather than writing GUI tests. This saves time and money, and helps bring developers and testers clo
ser together in the communication gap. The best solution here, IMO, is to add the service layer tool, which acts as a natural meeting place for developers and testers, but without the high costs of GUI tests.

So, sometimes, just based on trust and communication alone, you can lessen your GUI testing need for acceptance testing, and instead just communicate with the developers to make sure they are fully testing/exercising particular acceptance tests. A better solution is to downright prove that you don't need as much GUI testing because you and the devs have a highly readable service layer test (Cucumber, Fitnesse, etc) that proves that your acceptance tests are fully automated and exercised.

In short, you can develop and run service layer acceptance tests(Cucumber/Fitnesse), much cheaper and quicker than you can running GUI layer tests (Sellenium, etc). You can also exercise more data scenarios (same feature, different test inputs) much cheaper and quicker than with GUI test tools.

This is not to say that GUI test tools are bad, just that they should be the last resort(assuming you have a service layer tool), and should generally be used to test logic that resides actually in the GUI layer (which again, according to arch/design best practices, should be very little logic).

Also, I'm sort of assuming your developers also do unit/integration testing with JUnit or a similar tool. You mentioned that they use Sellenium, but I sure hope they're also doing a lot of JUnit stuff. Just like service layer testing is cheaper than GUI testing, unit/integration testing using JUnit is also cheaper than service layer testing(Cucumber, Fitnesse, etc).

-------
Charles Bradley, CSM, PSM I
Experienced Scrum Coach
My blog: http://scrumcrazy.wordpress.com/
Post by Conrad Ononeme
Hello ALL!
I have a quick question.
I am the lead embedded tester in my team that has done very well in the uptake of Agile.
Now having successfully gotten the team to follow best practice in pretty much every other area of Agile Software development, I now want to push the boat out a little by seeing if we can actually extend the developers' tests to form the basis (or at least part of it) of our Acceptance Tests.
------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/agile-testing/

<*> Your email settings:
Individual Email | Traditional

<*> To change settings online go to:
http://groups.yahoo.com/group/agile-testing/join
(Yahoo! ID required)

<*> To change settings via email:
agile-testing-digest-***@public.gmane.org
agile-testing-fullfeatured-***@public.gmane.org

<*> To unsubscribe from this group, send an email to:
agile-testing-unsubscribe-***@public.gmane.org

<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
John Goodsen
2011-03-01 02:22:59 UTC
Permalink
with over 50% of your code in the GUI layers, the argument to not test thru
the GUI becomes a cop out for writing and maintaining good acceptance
tests... google "declarative vs imperative cucumber" for more on that...

yes, I know I'm on the controversial side of that one...

(sent from my droid)
On Feb 28, 2011 1:29 PM, "charles_bradley_scrum_coach" <
Post by Markus Gaertner
Conrad,
You're on the right path, so keep up the good work.
http://blog.mountaingoatsoftware.com/the-forgotten-layer-of-the-test-automation-pyramid
Post by Markus Gaertner
Also read the comments specifically the one about the "Cloud" that resides
above the pyramid of "Exploratory Testing"
Post by Markus Gaertner
Everyone so far has given good advice, of which the theme, I believe, is
this: "Obtain and use a Service Layer testing tool like Cucumber, Fitnesse,
etc to support your GUI testing(Sellenium)"
Post by Markus Gaertner
My main justification for this, is the same as Cohn's -- once you get over
the learning curve with your service layer tool, tests to exercise the
service layer (where a huge percentage of business logic resides... or
SHOULD reside 'if your team is architecting/designing using best practices')
are much cheaper and easier to maintain than GUI/Sellenium tests.
Post by Markus Gaertner
Another suggestion: I've found that, when I coach teams on heading down
the "let's merge the testers' and developers' roads" like you are, an
interesting phenomenon happens. Often times, a particular piece of logic
will be well tested at the integration or unit level(which may or may not be
exposed to your service testing tool). When developers and testers
communicate about this, testers will find less need to test that logic at
the GUI level(which is essentially also what happens when you do service
layer testing -- less need to do GUI level testing) They'll still test, but
they'll usually just that logic in an exploratory way(just a few boundary
cases, rather than exhaustive testing) rather than writing GUI tests. This
saves time and money, and helps bring developers and testers closer together
in the communication gap. The best solution here, IMO, is to add the service
layer tool, which acts as a natural meeting place for developers and
testers, but without the high costs of GUI tests.
Post by Markus Gaertner
So, sometimes, just based on trust and communication alone, you can lessen
your GUI testing need for acceptance testing, and instead just communicate
with the developers to make sure they are fully testing/exercising
particular acceptance tests. A better solution is to downright prove that
you don't need as much GUI testing because you and the devs have a highly
readable service layer test (Cucumber, Fitnesse, etc) that proves that your
acceptance tests are fully automated and exercised.
Post by Markus Gaertner
In short, you can develop and run service layer acceptance
tests(Cucumber/Fitnesse), much cheaper and quicker than you can running GUI
layer tests (Sellenium, etc). You can also exercise more data scenarios
(same feature, different test inputs) much cheaper and quicker than with GUI
test tools.
Post by Markus Gaertner
This is not to say that GUI test tools are bad, just that they should be
the last resort(assuming you have a service layer tool), and should
generally be used to test logic that resides actually in the GUI layer
(which again, according to arch/design best practices, should be very little
logic).
Post by Markus Gaertner
Also, I'm sort of assuming your developers also do unit/integration
testing with JUnit or a similar tool. You mentioned that they use Sellenium,
but I sure hope they're also doing a lot of JUnit stuff. Just like service
layer testing is cheaper than GUI testing, unit/integration testing using
JUnit is also cheaper than service layer testing(Cucumber, Fitnesse, etc).
Post by Markus Gaertner
-------
Charles Bradley, CSM, PSM I
Experienced Scrum Coach
My blog: http://scrumcrazy.wordpress.com/
Post by Conrad Ononeme
Hello ALL!
I have a quick question.
I am the lead embedded tester in my team that has done very well in the uptake of Agile.
Now having successfully gotten the team to follow best practice in pretty
much every other area of Agile Software development, I now want to push the
boat out a little by seeing if we can actually extend the developers' tests
to form the basis (or at least part of it) of our Acceptance Tests.
Post by Markus Gaertner
------------------------------------
Yahoo! Groups Links
Matthew
2011-03-01 13:26:53 UTC
Permalink
Post by John Goodsen
with over 50% of your code in the GUI layers, the argument to not test thru
the GUI becomes a cop out for writing and maintaining good acceptance
tests... google "declarative vs imperative cucumber" for more on that...
yes, I know I'm on the controversial side of that one...
I suspect it really comes down to where your bugs come from. For example, at Socialtext, we have a 'watch page' feature. You click a star that says "watch page", the star lights up, then you clicked the "watched pages" link, the page now appears in the link. Go back, unwatch, watched pages, page does *not* appear.

That's not the kind of thing I want to test through the business logic layer.

Our /whole app/ is like that.

We do have javascript unit tests, but I find they are more helpful to the developers than to the testers -- a failure doesn't really indicate something is broken, a pass doesn't indicate it isn't ... they function more like change detectors.

We have also had a fair amount of success with GUI automation with Selenium, but a lot of that is due to the nature of our product and development process. In general, I recommend exploratory testing the GUI and a few small, light, quick, build verification GUI-driving tests.

Meanwhile, on the Software-Testing Discussion list, Elisabeth Hendrickson is making a big deal about how ATDD is /Acceptance/ Test Driven Development - not "Automated Testing During Development", and Ken Pugh is pointing out that you can do ATDD without automation:

http://groups.yahoo.com/group/software-testing/

Might be something to consider here, as, well, as I am /certainly/ not picking up that vibe here.

regards,
--
Matthew Heusser,
Personal Blog: http://xndev.blogspot.com/
Test Community Blog: http://softwaretestpro.com/blog/
Twitter: mheusser



------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/agile-testing/

<*> Your email settings:
Individual Email | Traditional

<*> To change settings online go to:
http://groups.yahoo.com/group/agile-testing/join
(Yahoo! ID required)

<*> To change settings via email:
agile-testing-digest-***@public.gmane.org
agile-testing-fullfeatured-***@public.gmane.org

<*> To unsubscribe from this group, send an email to:
agile-testing-unsubscribe-***@public.gmane.org

<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
c***@public.gmane.org
2011-03-01 14:11:00 UTC
Permalink
Thanks for this new perspective, Matthew. It looks certainly very
interesting and I am going to check the link out straight away.

I have kind of decided that we will drive the ATDD for my team via
cucumber (which i am currently reading up on now as a direct result of
all the very helpful suggestions I have received from one and all)

I am very grateful for all your responses which has helped me a great
deal in grappling with my 'beast', and a BIG Thank you to everyone that
pitched in to help with their wonderful ideas as always!

I will revert with further questions I am sure, in due course!

Warmest Regards

C O


-----Original Message-----
From: Matthew <matt.heusser-***@public.gmane.org>
To: agile-testing-***@public.gmane.org
Sent: Tue, Mar 1, 2011 2:26 pm
Subject: [agile-testing] Re: Extending TDD to ATDD
Post by John Goodsen
with over 50% of your code in the GUI layers, the argument to not test thru
the GUI becomes a cop out for writing and maintaining good acceptance
tests... google "declarative vs imperative cucumber" for more on that...
yes, I know I'm on the controversial side of that one...
I suspect it really comes down to where your bugs come from. For
example, at Socialtext, we have a 'watch page' feature. You click a
star that says "watch page", the star lights up, then you clicked the
"watched pages" link, the page now appears in the link. Go back,
unwatch, watched pages, page does *not* appear.

That's not the kind of thing I want to test through the business logic
layer.

Our /whole app/ is like that.

We do have javascript unit tests, but I find they are more helpful to
the developers than to the testers -- a failure doesn't really indicate
something is broken, a pass doesn't indicate it isn't ... they function
more like change detectors.

We have also had a fair amount of success with GUI automation with
Selenium, but a lot of that is due to the nature of our product and
development process. In general, I recommend exploratory testing the
GUI and a few small, light, quick, build verification GUI-driving
tests.

Meanwhile, on the Software-Testing Discussion list, Elisabeth
Hendrickson is making a big deal about how ATDD is /Acceptance/ Test
Driven Development - not "Automated Testing During Development", and
Ken Pugh is pointing out that you can do ATDD without automation:

http://groups.yahoo.com/group/software-testing/

Might be something to consider here, as, well, as I am /certainly/ not
picking up that vibe here.

regards,

--
Matthew Heusser,
Personal Blog: http://xndev.blogspot.com/
Test Community Blog: http://softwaretestpro.com/blog/
Twitter: mheusser









------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/agile-testing/

<*> Your email settings:
Individual Email | Traditional

<*> To change settings online go to:
http://groups.yahoo.com/group/agile-testing/join
(Yahoo! ID required)

<*> To change settings via email:
agile-testing-digest-***@public.gmane.org
agile-testing-fullfeatured-***@public.gmane.org

<*> To unsubscribe from this group, send an email to:
agile-testing-unsubscribe-***@public.gmane.org

<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
Janet Gregory
2011-03-01 14:19:40 UTC
Permalink
Both Elisabeth and Ken are correct about ATDD. It is a process. I suggest teams use the process if they don't have the automation in place, so they can get the biggest benefit. Using collaborative tools to automate during the process is a great side-effect but not the main purpose.

Janet

Janet Gregory
Co-author with Lisa Crispin, Agile Testing: A Practical Guide for Testers and Agile Teams
DragonFire Inc.
www.janetgregory.ca

----- Original Message -----
From: Matthew <matt.heusser-***@public.gmane.org>
Date: Tuesday, March 1, 2011 6:26 am
Subject: [agile-testing] Re: Extending TDD to ATDD
Post by John Goodsen
Post by John Goodsen
with over 50% of your code in the GUI layers, the argument to
not test thru
Post by John Goodsen
the GUI becomes a cop out for writing and maintaining good acceptance
tests... google "declarative vs imperative cucumber" for more
on that...
Post by John Goodsen
yes, I know I'm on the controversial side of that one...
I suspect it really comes down to where your bugs come
from. For example, at Socialtext, we have a 'watch page'
feature. You click a star that says "watch page", the star
lights up, then you clicked the "watched pages" link, the page
now appears in the link. Go back, unwatch, watched pages,
page does *not* appear.
That's not the kind of thing I want to test through the business logic layer.
Our /whole app/ is like that.
We do have javascript unit tests, but I find they are more
helpful to the developers than to the testers -- a failure
doesn't really indicate something is broken, a pass doesn't
indicate it isn't ... they function more like change detectors.
We have also had a fair amount of success with GUI automation
with Selenium, but a lot of that is due to the nature of our
product and development process. In general, I recommend
exploratory testing the GUI and a few small, light, quick, build
verification GUI-driving tests.
Meanwhile, on the Software-Testing Discussion list, Elisabeth
Hendrickson is making a big deal about how ATDD is /Acceptance/
Test Driven Development - not "Automated Testing During
Development", and Ken Pugh is pointing out that you can do ATDD
http://groups.yahoo.com/group/software-testing/
Might be something to consider here, as, well, as I am
/certainly/ not picking up that vibe here.
regards,
--
Matthew Heusser,
Personal Blog: http://xndev.blogspot.com/
Test Community Blog: http://softwaretestpro.com/blog/
Twitter: mheusser
Rohan Sarker
2011-03-01 14:51:06 UTC
Permalink
*Engineering Mechanics Representations gives a better picture of TDD -- >
ATDD*
*
*
***For instance, Indian Developers / Testers (I have seen in my practical
experience) that they use the Most Frequently Asked Question :: How many
persons were there in the Process and use the answer to determine the
coefficients of the Equation.......... ;-<*
*
*
*
*
*Where you were then Questions - Answer is used to Locate the person's
[MODEL] in the different components of the Equation :-(
*
*
*
*
*
*
*
***Regards*
*Rohan Sarker*
*+917278539338*
*+913324288069
*
Post by Janet Gregory
Both Elisabeth and Ken are correct about ATDD. It is a process. I suggest
teams use the process if they don't have the automation in place, so they
can get the biggest benefit. Using collaborative tools to automate during
the process is a great side-effect but not the main purpose.
Janet
Janet Gregory
Co-author with Lisa Crispin, Agile Testing: A Practical Guide for Testers and Agile Teams
DragonFire Inc.
www.janetgregory.ca
----- Original Message -----
Date: Tuesday, March 1, 2011 6:26 am
Subject: [agile-testing] Re: Extending TDD to ATDD
Post by John Goodsen
Post by John Goodsen
with over 50% of your code in the GUI layers, the argument to
not test thru
Post by John Goodsen
the GUI becomes a cop out for writing and maintaining good acceptance
tests... google "declarative vs imperative cucumber" for more
on that...
Post by John Goodsen
yes, I know I'm on the controversial side of that one...
I suspect it really comes down to where your bugs come
from. For example, at Socialtext, we have a 'watch page'
feature. You click a star that says "watch page", the star
lights up, then you clicked the "watched pages" link, the page
now appears in the link. Go back, unwatch, watched pages,
page does *not* appear.
That's not the kind of thing I want to test through the business logic layer.
Our /whole app/ is like that.
We do have javascript unit tests, but I find they are more
helpful to the developers than to the testers -- a failure
doesn't really indicate something is broken, a pass doesn't
indicate it isn't ... they function more like change detectors.
We have also had a fair amount of success with GUI automation
with Selenium, but a lot of that is due to the nature of our
product and development process. In general, I recommend
exploratory testing the GUI and a few small, light, quick, build
verification GUI-driving tests.
Meanwhile, on the Software-Testing Discussion list, Elisabeth
Hendrickson is making a big deal about how ATDD is /Acceptance/
Test Driven Development - not "Automated Testing During
Development", and Ken Pugh is pointing out that you can do ATDD
http://groups.yahoo.com/group/software-testing/
Might be something to consider here, as, well, as I am
/certainly/ not picking up that vibe here.
regards,
--
Matthew Heusser,
Personal Blog: http://xndev.blogspot.com/
Test Community Blog: http://softwaretestpro.com/blog/
Twitter: mheusser
George Dinwiddie
2011-03-01 22:37:38 UTC
Permalink
Matt,
Post by Matthew
I suspect it really comes down to where your bugs come from. For
example, at Socialtext, we have a 'watch page' feature. You click a
star that says "watch page", the star lights up, then you clicked the
"watched pages" link, the page now appears in the link. Go back,
unwatch, watched pages, page does *not* appear.
That's not the kind of thing I want to test through the business logic layer.
If the page watching logic is in the business logic layer and the
star-light and watched pages list are in the model that layer provides
to the GUI, then why not?

I'd also want to test that the GUI is hooked up correctly to the data
model and the actions of the business logic layer, of course.

- George
--
----------------------------------------------------------------------
* George Dinwiddie * http://blog.gdinwiddie.com
Software Development http://www.idiacomputing.com
Consultant and Coach http://www.agilemaryland.org
----------------------------------------------------------------------



------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/agile-testing/

<*> Your email settings:
Individual Email | Traditional

<*> To change settings online go to:
http://groups.yahoo.com/group/agile-testing/join
(Yahoo! ID required)

<*> To change settings via email:
agile-testing-digest-***@public.gmane.org
agile-testing-fullfeatured-***@public.gmane.org

<*> To unsubscribe from this group, send an email to:
agile-testing-unsubscribe-***@public.gmane.org

<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
Matthew
2011-03-02 13:27:49 UTC
Permalink
Post by George Dinwiddie
Matt,
Post by Matthew
I suspect it really comes down to where your bugs come from. For
example, at Socialtext, we have a 'watch page' feature. You click a
star that says "watch page", the star lights up, then you clicked the
"watched pages" link, the page now appears in the link. Go back,
unwatch, watched pages, page does *not* appear.
That's not the kind of thing I want to test through the business logic layer.
If the page watching logic is in the business logic layer and the
star-light and watched pages list are in the model that layer provides
to the GUI, then why not?
I'd also want to test that the GUI is hooked up correctly to the data
model and the actions of the business logic layer, of course.
I was really disappointed by this comment, George. I struggled for some time to come up with a reply that was helpful. That said, here goes ...

I think the most polite thing I can say here is that you are speaking in abstractions that don't map well onto a LAMP/AJAX technology infrastructure. I mean, you could /try/ to separate concerns in this way, and just test at the business logic level, but you'd /fail/ -- consider testing a WYSIWYG customer control for bold and unbold -- those things don't even /make/ server trips.

Even if you could find a way to test watch, sooner or later, it would break and the business logic layer would be just fine. This has actually happened.

Not only that, the idea that one little change "couldn't possible break" the system and there is "no need to look over there" for the problem is /classic/. I believe Jerry Weinberg uses it in "Becoming a Technical Leader", but it might be "Secrets of Consulting": His general conclusion is that when people say "the bug couldn't possibly be in there" you should *look in there*.

Another short reason might be because 'just' testing the business logic for a GUI feature isn't testing the whole black box. To quote Bob Martin: "End to end is further than you think."

Of course, it's possible to 'cover' for not writing selenium tests in several ways. (or 'test it twice') You could do exploratory tests or have a 'beta'/staging program to generate coverage of the application, etc.

Your email didn't speak to those other ways of covering the app; instead, you asked "why not" "just" test at the Business Logic Level.

/*Because you'll miss the bugs, dude!*/

Asked and answered.

regards,
--
Matthew Heusser,
Personal Blog: http://xndev.blogspot.com/
Test Community Blog: http://softwaretestpro.com/blog/
Twitter: mheusser




------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/agile-testing/

<*> Your email settings:
Individual Email | Traditional

<*> To change settings online go to:
http://groups.yahoo.com/group/agile-testing/join
(Yahoo! ID required)

<*> To change settings via email:
agile-testing-digest-***@public.gmane.org
agile-testing-fullfeatured-***@public.gmane.org

<*> To unsubscribe from this group, send an email to:
agile-testing-unsubscribe-***@public.gmane.org

<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
charles_bradley_scrum_coach
2011-03-04 23:42:07 UTC
Permalink
Matt,

As I'm reading back over your response, it dawns on me.
Post by Matthew
Your email didn't speak to those other ways of covering the app;
instead, you asked "why not" "just" test at the Business Logic
Level.
I'd also want to test that the GUI is hooked up correctly to the
data model and the actions of the business logic layer, of course.
Does this statement by George above not address your concern that people suggest to "skip GUI testing" altogether?

I wasn't saying to skip GUI testing, and I don't think George said that, either. I think what he and I (and Mike Cohn) were saying is that it might be wise to test *less* at the GUI layer *if* you do good service level testing, since service level tests are cheaper and easier to maintain(again, once you get over the not so big learning curve of your business service layer testing tool/s)

-------
Charles Bradley, CSM, PSM I
Experienced Scrum Coach
My blog: http://scrumcrazy.wordpress.com/
Post by Matthew
Post by George Dinwiddie
Matt,
Post by Matthew
I suspect it really comes down to where your bugs come from. For
example, at Socialtext, we have a 'watch page' feature. You click a
star that says "watch page", the star lights up, then you clicked the
"watched pages" link, the page now appears in the link. Go back,
unwatch, watched pages, page does *not* appear.
That's not the kind of thing I want to test through the business logic layer.
If the page watching logic is in the business logic layer and the
star-light and watched pages list are in the model that layer provides
to the GUI, then why not?
I'd also want to test that the GUI is hooked up correctly to the data
model and the actions of the business logic layer, of course.
I was really disappointed by this comment, George. I struggled for some time to come up with a reply that was helpful. That said, here goes ...
I think the most polite thing I can say here is that you are speaking in abstractions that don't map well onto a LAMP/AJAX technology infrastructure. I mean, you could /try/ to separate concerns in this way, and just test at the business logic level, but you'd /fail/ -- consider testing a WYSIWYG customer control for bold and unbold -- those things don't even /make/ server trips.
Even if you could find a way to test watch, sooner or later, it would break and the business logic layer would be just fine. This has actually happened.
Not only that, the idea that one little change "couldn't possible break" the system and there is "no need to look over there" for the problem is /classic/. I believe Jerry Weinberg uses it in "Becoming a Technical Leader", but it might be "Secrets of Consulting": His general conclusion is that when people say "the bug couldn't possibly be in there" you should *look in there*.
Another short reason might be because 'just' testing the business logic for a GUI feature isn't testing the whole black box. To quote Bob Martin: "End to end is further than you think."
Of course, it's possible to 'cover' for not writing selenium tests in several ways. (or 'test it twice') You could do exploratory tests or have a 'beta'/staging program to generate coverage of the application, etc.
Your email didn't speak to those other ways of covering the app; instead, you asked "why not" "just" test at the Business Logic Level.
/*Because you'll miss the bugs, dude!*/
Asked and answered.
regards,
--
Matthew Heusser,
Personal Blog: http://xndev.blogspot.com/
Test Community Blog: http://softwaretestpro.com/blog/
Twitter: mheusser
------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/agile-testing/

<*> Your email settings:
Individual Email | Traditional

<*> To change settings online go to:
http://groups.yahoo.com/group/agile-testing/join
(Yahoo! ID required)

<*> To change settings via email:
agile-testing-digest-***@public.gmane.org
agile-testing-fullfeatured-***@public.gmane.org

<*> To unsubscribe from this group, send an email to:
agile-testing-unsubscribe-***@public.gmane.org

<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
Matthew
2011-03-05 14:40:16 UTC
Permalink
Post by George Dinwiddie
I'd also want to test that the GUI is hooked up correctly to the
data model and the actions of the business logic layer, of course.
Point taken, Charles.

Then again, when I see an Ad with 30 seconds about sugar smacks and 3 seconds about "part of a balanced breakfast", I don't see that as actually promoting a balanced breakfast.

Testing is a /skill/. It takes practice and learning. Seeing it minimized as "oh yeah, you gotta do end to end testing too" is disappointing.

The comparison of breakfast choice to businesss layer and end to end tests is (mostly) an exercise for the reader. :-)


regards,
--
Matthew Heusser,
Personal Blog: http://xndev.blogspot.com/
Test Community Blog: http://softwaretestpro.com/blog/
Twitter: mheusser





------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/agile-testing/

<*> Your email settings:
Individual Email | Traditional

<*> To change settings online go to:
http://groups.yahoo.com/group/agile-testing/join
(Yahoo! ID required)

<*> To change settings via email:
agile-testing-digest-***@public.gmane.org
agile-testing-fullfeatured-***@public.gmane.org

<*> To unsubscribe from this group, send an email to:
agile-testing-unsubscribe-***@public.gmane.org

<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
Charles Bradley - Scrum Coach CSM PSM I
2011-03-05 16:52:32 UTC
Permalink
Your point taken as well.

I'm a big believer in two main themes:
1. When designing/architecting, push the logic down as far as it can feasibly
go.
2. Focus most testing in the layer that the logic resides, then also do
proportionately less testing end to end to make sure that logic works in the
whole as well.

Some have made comments here like "test where most of the bugs are". I suggest
also to "test where the logic is".


If a system has a whole bunch of logic in the GUI, then I say two things:
1. Test the heck out of the GUI, because it has a bunch of logic in it.
2. Have a conversation with someone who can influence pushing the logic to a
(feasible)lower layer.

I don't mean to minimize GUI testing, only to suggest that over the long hall,
GUI testing costs > Service testing costs > Code(JUnit, etc) testing costs. As
such, try to minimize costs while maximizing test/logic coverage.

-------
Charles Bradley, CSM, PSM I
Experienced Scrum Coach
My blog: http://scrumcrazy.wordpress.com/




________________________________
From: Matthew <matt.heusser-***@public.gmane.org>
To: agile-testing-***@public.gmane.org
Sent: Sat, March 5, 2011 7:40:16 AM
Subject: [agile-testing] Re: Extending TDD to ATDD
Post by George Dinwiddie
I'd also want to test that the GUI is hooked up correctly to the
data model and the actions of the business logic layer, of course.
Point taken, Charles.

Then again, when I see an Ad with 30 seconds about sugar smacks and 3 seconds
about "part of a balanced breakfast", I don't see that as actually promoting a
balanced breakfast.

Testing is a /skill/. It takes practice and learning. Seeing it minimized as
"oh yeah, you gotta do end to end testing too" is disappointing.

The comparison of breakfast choice to businesss layer and end to end tests is
(mostly) an exercise for the reader. :-)

regards,
--
Matthew Heusser,
Personal Blog: http://xndev.blogspot.com/
Test Community Blog: http://softwaretestpro.com/blog/
Twitter: mheusser
Josue Barbosa dos Santos
2011-03-02 16:25:08 UTC
Permalink
I suspect it really comes down to where your bugs come from. For example,
at Socialtext, we have a 'watch page' feature. You click >>a star that says
"watch page", the star lights up, then you clicked the "watched pages" link,
the page now appears in the link. Go >>back, unwatch, watched pages, page
does *not* appear.
That's not the kind of thing I want to test through the business logic layer.
May be when he says business logic layer in fact he is meaning the
possibility to write unit tests (micro tests) to the logic without the need
of tools like selenium. And using a pattern like MVP it is possible to the
example you gave. Ex:

....
//This pseudo-code is just to give an idea
@Test
whenClickWatchPageShouldStarLightsUpAndAddThePageToWatchedPages(){
view = context.mock(View)
model=context.mock(model)
Presenter p = new Presenter(view, model)
context.Expects{
view.starLightsUp(page)
model.addToWatched(page)
}
//this should be called on appropriate element on view
presenter.watch(page)
}

@Test
whenClickOnWatchedPagesShouldShowTheWatchedPages(){
view = context.mock(View)
model=context.mock(model)
Presenter p = new Presenter(view, model)
context.Expects{
model.getWatchedPages();willReturn(page1,page2)
view.addToWatched(page1)
view.addToWatched(page2)
}
//this should be called on appropriate element on view
presenter.showWatchedPages()
}

etc...

And one could be comfortable to not write GUI tests to this logic. Only
doing some exploratory to see if everything is ok.

Note I am not saying that it is the right approach. It is one approach. Here
we write BOTH tests.But we try to write code in a way that when an error
occur in an web test, one unit test should also be broken. If it not happens
we investigate to see if it is possible to write an equivalent unit test.

By the way, it seems that Joshua Kerievsky and Adam Sroka are not using
Acceptance Tests at all. Only unit tests in their projects. I think I read
it in the TDD list. Not sure.

--
Abraços,
Josué
http://twitter.com/josuesantos
with over 50% of your code in the GUI layers, the argument to not test
thru
the GUI becomes a cop out for writing and maintaining good acceptance
tests... google "declarative vs imperative cucumber" for more on that...
yes, I know I'm on the controversial side of that one...
I suspect it really comes down to where your bugs come from. For example,
at Socialtext, we have a 'watch page' feature. You click a star that says
"watch page", the star lights up, then you clicked the "watched pages" link,
the page now appears in the link. Go back, unwatch, watched pages, page does
*not* appear.
That's not the kind of thing I want to test through the business logic layer.
Our /whole app/ is like that.
We do have javascript unit tests, but I find they are more helpful to the
developers than to the testers -- a failure doesn't really indicate
something is broken, a pass doesn't indicate it isn't ... they function more
like change detectors.
We have also had a fair amount of success with GUI automation with
Selenium, but a lot of that is due to the nature of our product and
development process. In general, I recommend exploratory testing the GUI and
a few small, light, quick, build verification GUI-driving tests.
Meanwhile, on the Software-Testing Discussion list, Elisabeth Hendrickson
is making a big deal about how ATDD is /Acceptance/ Test Driven Development
- not "Automated Testing During Development", and Ken Pugh is pointing out
http://groups.yahoo.com/group/software-testing/
Might be something to consider here, as, well, as I am /certainly/ not
picking up that vibe here.
regards,
--
Matthew Heusser,
Personal Blog: http://xndev.blogspot.com/
Test Community Blog: http://softwaretestpro.com/blog/
Twitter: mheusser
--
Abraços,
Josué
http://twitter.com/josuesantos
Matthew
2011-03-02 19:25:22 UTC
Permalink
Post by Josue Barbosa dos Santos
May be when he says business logic layer in fact he is meaning the
possibility to write unit tests (micro tests) to the logic without the need
of tools like selenium. And using a pattern like MVP it is possible to the
Thanks Jose. Yes, that was exactly what I assumed he meant. I'm familiar with mocks; I gave a talk with Sean McMcmillan on the subject at Google Test Automation Conference a few years back:



Notice Sean McMillan's concerns about 15:00-16:12 in. Also notice my comments immediately following that. I am still expressing those concerns today. (As I've mentioned on or twice all ready, Micro-tests over time in Javascript tend to act more like change detectors. "Failure" means some expectation no longer holds true, "passing" means it still holds true - but both of those cases might or might involve new defects.)

As to the claim that developer-tests alone can be sufficient for testing, well, I would assert "not where I work." I /can/ see /some/ domains where that can work fine, such as applications with very light javascript designed to run in a single browser for internal company use, etc.

I work at a company deploying software as a service for a thousands of professional customers supporting a large variety of browsers and mobile devices that provides a fair amount of client-side "eye candy." The context are different. (I talk about applying a strategy that works in one context into an incorrect context at 24:24 :-)

Thank you for the trip down memory lane; I'm pleased that the talk holds up as well as it does. :-)

regards,
--
Matthew Heusser,
Personal Blog: http://xndev.blogspot.com/
Test Community Blog: http://softwaretestpro.com/blog/
Twitter: mheusser





------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/agile-testing/

<*> Your email settings:
Individual Email | Traditional

<*> To change settings online go to:
http://groups.yahoo.com/group/agile-testing/join
(Yahoo! ID required)

<*> To change settings via email:
agile-testing-digest-***@public.gmane.org
agile-testing-fullfeatured-***@public.gmane.org

<*> To unsubscribe from this group, send an email to:
agile-testing-unsubscribe-***@public.gmane.org

<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
charles_bradley_scrum_coach
2011-03-01 18:47:22 UTC
Permalink
John,

I'm not sure what you're saying here.

Are you saying that, if one has 50% of their code in GUI layers, then one should not just skip GUI tests and do only service layer testing?

Is that what you're saying?

I did google the declarative vs. imperative, but all I came up with was two different styles of expressing user story acceptance tests, and two different styles of writing cucumber tests. Could you expand on how this is relevant to what I posted?

-------
Charles Bradley, CSM, PSM I
Experienced Scrum Coach
My blog: http://scrumcrazy.wordpress.com/
Post by John Goodsen
with over 50% of your code in the GUI layers, the argument to not test thru
the GUI becomes a cop out for writing and maintaining good acceptance
tests... google "declarative vs imperative cucumber" for more on that...
yes, I know I'm on the controversial side of that one...
(sent from my droid)
On Feb 28, 2011 1:29 PM, "charles_bradley_scrum_coach" <
Post by Markus Gaertner
Conrad,
You're on the right path, so keep up the good work.
First, if you're not already familiar with the "Testing Pyramid" concept,
http://blog.mountaingoatsoftware.com/the-forgotten-layer-of-the-test-automation-pyramid
Post by Markus Gaertner
Also read the comments specifically the one about the "Cloud" that resides
above the pyramid of "Exploratory Testing"
Post by Markus Gaertner
Everyone so far has given good advice, of which the theme, I believe, is
this: "Obtain and use a Service Layer testing tool like Cucumber, Fitnesse,
etc to support your GUI testing(Sellenium)"
Post by Markus Gaertner
My main justification for this, is the same as Cohn's -- once you get over
the learning curve with your service layer tool, tests to exercise the
service layer (where a huge percentage of business logic resides... or
SHOULD reside 'if your team is architecting/designing using best practices')
are much cheaper and easier to maintain than GUI/Sellenium tests.
------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/agile-testing/

<*> Your email settings:
Individual Email | Traditional

<*> To change settings online go to:
http://groups.yahoo.com/group/agile-testing/join
(Yahoo! ID required)

<*> To change settings via email:
agile-testing-digest-***@public.gmane.org
agile-testing-fullfeatured-***@public.gmane.org

<*> To unsubscribe from this group, send an email to:
agile-testing-unsubscribe-***@public.gmane.org

<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
Markus Gaertner
2011-03-02 07:40:05 UTC
Permalink
Hi Charles,
Another suggestion: I've found that, when I coach teams on heading down the "let's merge the testers' and developers' roads" like you are, an interesting phenomenon happens. Often times, a particular piece of logic will be well tested at the integration or unit level(which may or may not be exposed to your service testing tool). When developers and testers communicate about this, testers will find less need to test that logic at the GUI level(which is essentially also what happens when you do service layer testing -- less need to do GUI level testing) They'll still test, but they'll usually just that logic in an exploratory way(just a few boundary cases, rather than exhaustive testing) rather than writing GUI tests. This saves time and money, and helps bring developers and testers closer together in the communication gap. The best solution here, IMO, is to add the service layer tool, which acts as a natural meeting place for developers and testers, but without the high costs of GUI tests.
???

I love this paragraph.

Best
Markus Gärtner
http://www.shino.de/blog
http://www.it-agile.de
@mgaertne
Conrad,
You're on the right path, so keep up the good work.
http://blog.mountaingoatsoftware.com/the-forgotten-layer-of-the-test-automation-pyramid
Also read the comments specifically the one about the "Cloud" that resides above the pyramid of "Exploratory Testing"
Everyone so far has given good advice, of which the theme, I believe, is this: "Obtain and use a Service Layer testing tool like Cucumber, Fitnesse, etc to support your GUI testing(Sellenium)"
My main justification for this, is the same as Cohn's -- once you get over the learning curve with your service layer tool, tests to exercise the service layer (where a huge percentage of business logic resides... or SHOULD reside 'if your team is architecting/designing using best practices') are much cheaper and easier to maintain than GUI/Sellenium tests.
Another suggestion: I've found that, when I coach teams on heading down the "let's merge the testers' and developers' roads" like you are, an interesting phenomenon happens. Often times, a particular piece of logic will be well tested at the integration or unit level(which may or may not be exposed to your service testing tool). When developers and testers communicate about this, testers will find less need to test that logic at the GUI level(which is essentially also what happens when you do service layer testing -- less need to do GUI level testing) They'll still test, but they'll usually just that logic in an exploratory way(just a few boundary cases, rather than exhaustive testing) rather than writing GUI tests. This saves time and money, and helps bring developers and testers closer together in the communication gap. The best solution here, IMO, is to add the service layer tool, which acts as a natural meeting place for developers and testers, but without the high costs of GUI tests.
So, sometimes, just based on trust and communication alone, you can lessen your GUI testing need for acceptance testing, and instead just communicate with the developers to make sure they are fully testing/exercising particular acceptance tests. A better solution is to downright prove that you don't need as much GUI testing because you and the devs have a highly readable service layer test (Cucumber, Fitnesse, etc) that proves that your acceptance tests are fully automated and exercised.
In short, you can develop and run service layer acceptance tests(Cucumber/Fitnesse), much cheaper and quicker than you can running GUI layer tests (Sellenium, etc). You can also exercise more data scenarios (same feature, different test inputs) much cheaper and quicker than with GUI test tools.
This is not to say that GUI test tools are bad, just that they should be the last resort(assuming you have a service layer tool), and should generally be used to test logic that resides actually in the GUI layer (which again, according to arch/design best practices, should be very little logic).
Also, I'm sort of assuming your developers also do unit/integration testing with JUnit or a similar tool. You mentioned that they use Sellenium, but I sure hope they're also doing a lot of JUnit stuff. Just like service layer testing is cheaper than GUI testing, unit/integration testing using JUnit is also cheaper than service layer testing(Cucumber, Fitnesse, etc).
-------
Charles Bradley, CSM, PSM I
Experienced Scrum Coach
My blog: http://scrumcrazy.wordpress.com/
Post by Conrad Ononeme
Hello ALL!
I have a quick question.
I am the lead embedded tester in my team that has done very well in the uptake of Agile.
Now having successfully gotten the team to follow best practice in pretty much every other area of Agile Software development, I now want to push the boat out a little by seeing if we can actually extend the developers' tests to form the basis (or at least part of it) of our Acceptance Tests.
------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/agile-testing/

<*> Your email settings:
Individual Email | Traditional

<*> To change settings online go to:
http://groups.yahoo.com/group/agile-testing/join
(Yahoo! ID required)

<*> To change settings via email:
agile-testing-digest-***@public.gmane.org
agile-testing-fullfeatured-***@public.gmane.org

<*> To unsubscribe from this group, send an email to:
agile-testing-unsubscribe-***@public.gmane.org

<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
Matthew
2011-03-02 13:32:36 UTC
Permalink
Post by Markus Gaertner
Hi Charles,
Post by charles_bradley_scrum_coach
Another suggestion: I've found that, when I coach teams on heading down the
"let's merge the testers' and developers' roads" like you are, an interesting
phenomenon happens. Often times, a particular piece of logic will be well tested
at the integration or unit level(which may or may not be exposed to your
service testing tool). When developers and testers communicate about this,
testers will find less need to test that logic at the GUI level(which is essentially
also what happens when you do service layer testing -- less need to do GUI
level testing) They'll still test, but they'll usually just that logic in an
exploratory way(just a few boundary cases, rather than exhaustive testing)
Like I said before, it depends on how much eye-candy is in the GUI, but I /can/ see this working on certain circumstances. I especially appreciate that you mentioned 'covering' the risks in other ways, eg exploratory testing. That's an important piece that's easy to miss.


regards,
--
Matthew Heusser,
Personal Blog: http://xndev.blogspot.com/
Test Community Blog: http://softwaretestpro.com/blog/
Twitter: mheusser



------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/agile-testing/

<*> Your email settings:
Individual Email | Traditional

<*> To change settings online go to:
http://groups.yahoo.com/group/agile-testing/join
(Yahoo! ID required)

<*> To change settings via email:
agile-testing-digest-***@public.gmane.org
agile-testing-fullfeatured-***@public.gmane.org

<*> To unsubscribe from this group, send an email to:
agile-testing-unsubscribe-***@public.gmane.org

<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
Mark Levison
2011-03-07 14:28:39 UTC
Permalink
Post by Markus Gaertner
Hi Charles,
Post by Markus Gaertner
Another suggestion: I've found that, when I coach teams on heading down
the "let's merge the testers' and developers' roads" like you are, an
interesting phenomenon happens. Often times, a particular piece of logic
will be well tested at the integration or unit level(which may or may not be
exposed to your service testing tool). When developers and testers
communicate about this, testers will find less need to test that logic at
the GUI level(which is essentially also what happens when you do service
layer testing -- less need to do GUI level testing) They'll still test, but
they'll usually just that logic in an exploratory way(just a few boundary
cases, rather than exhaustive testing) rather than writing GUI tests. This
saves time and money, and helps bring developers and testers closer together
in the communication gap. The best solution here, IMO, is to add the service
layer tool, which acts as a natural meeting place for developers and
testers, but without the high costs of GUI tests.
I assume in this last part Charles is just saying what most of us say: test
underneath the GUI as much as possible. In these cases the tests often wind
up describing a form of public API.

Cheers
Mark Levison

[image: Mark] <http://www.flickr.com/photos/***@N00/3833840021/>*Mark
Levison* | Agile Pain Relief Consulting <http://agilepainrelief.com/> |
Certified Scrum Trainer
Agile Editor @ InfoQ <http://www.infoq.com/about.jsp> |
Blog<http://www.notesfromatooluser.com/>|
Twitter <http://twitter.com/mlevison> | Office: (613) 862-2538
Recent Entries: Story Slicing How Small is Small
Enough<http://agilepainrelief.com/notesfromatooluser/2010/09/story-slicing-how-small-is-enough.html>,
Why use an Agile
Coach<http://www.notesfromatooluser.com/2009/11/why-use-an-agile-coach.html>
Charles Bradley - Scrum Coach CSM PSM I
2011-03-07 14:38:38 UTC
Permalink
Mark,
I assume in this last part Charles is just saying what most of us say: test
underneath the GUI as much as possible.
Yes, agreed, except I did add that part about "testers and developers" coming
together in a "natural meeting place" we sometimes call "service layer
testing". I think this "natural meeting place" is a good way for teams to
become more efficient and more cross functional(a la Scrum), which adds a whole
new set of cost savings. The "natural meeting place" was speaking more to team
dynamics than test efficiency.
In these cases the tests often wind up describing a form of public API.
I think I agree with everything here except "public". To me, "public" API
implies that you are planning to let other teams use your API directly. IMO,
much more thought has to be given to an API before one makes it "public."

And, FWIW, I did respond to Markus about quoting me, I just did so off-list.

-------
Charles Bradley, CSM, PSM I
Experienced Scrum Coach
My blog: http://scrumcrazy.wordpress.com/
Mark Levison
2011-03-07 14:50:46 UTC
Permalink
On Mon, Mar 7, 2011 at 9:38 AM, Charles Bradley - Scrum Coach CSM PSM I <
Mark,
test underneath the GUI as much as possible.
Yes, agreed, except I did add that part about "testers and developers"
coming together in a "natural meeting place" we sometimes call "service
layer testing". I think this "natural meeting place" is a good way for
teams to become more efficient and more cross functional(a la Scrum), which
adds a whole new set of cost savings. The "natural meeting place" was
speaking more to team dynamics than test efficiency.
I guess a better statement of my point would be that I was trying to bridge
the gap with Matt. Too most clients (who's app isn't mostly JavaScript), I
recommend

- 20-25% of their acceptance tests through the GUI
- the remainder underneath the GUI

As an example my current client is testing their advanced search
functionality. Its largely business logic and so I say test it through the
business layer for the most part. However the GUI should still be tested to
ensure that: things that can go wrong there are tested and that its
correctly wired. This tradeoff gives them a set of tests that run quickly
(minutes) vs. the GUI only tests of yore (hours).
In these cases the tests often wind up describing a form of public API.
I think I agree with everything here except "public". To me, "public" API
implies that you are planning to let other teams use your API directly.
IMO, much more thought has to be given to an API before one makes it
"public."
Hence my use of the word "form". It maybe a public API but you're giving
people access to it. Example said search functions are written in C#, the
application now exposes them as Objects/Methods. Other applications in the
client's stack could make use of this API, however you couldn't because you
will never have access to the process to speak to the API.

Cheers
Mark Levison

[image: Mark] <http://www.flickr.com/photos/***@N00/3833840021/>*Mark
Levison* | Agile Pain Relief Consulting <http://agilepainrelief.com/> |
Certified Scrum Trainer
Agile Editor @ InfoQ <http://www.infoq.com/about.jsp> |
Blog<http://www.notesfromatooluser.com/>|
Twitter <http://twitter.com/mlevison> | Office: (613) 862-2538
Recent Entries: Story Slicing How Small is Small
Enough<http://agilepainrelief.com/notesfromatooluser/2010/09/story-slicing-how-small-is-enough.html>,
Why use an Agile
Coach<http://www.notesfromatooluser.com/2009/11/why-use-an-agile-coach.html>
Loading...