<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii">
<META NAME="Generator" CONTENT="MS Exchange Server version 6.5.7651.59">
<TITLE>Testing a web API with twill?</TITLE>
</HEAD>
<BODY>
<!-- Converted from text/rtf format -->
<P DIR=LTR><SPAN LANG="en-us"><FONT FACE="Calibri">Hi</FONT></SPAN><SPAN LANG="en-us"> <FONT FACE="Calibri">–</FONT></SPAN><SPAN LANG="en-us"></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT FACE="Calibri">I’m trying to find a web unit and load testing tool that will let me test URL-based APIs</FONT></SPAN><SPAN LANG="en-us"><FONT FACE="Calibri">. This is basically a parameterized URL that will return JSON-encoded results.</FONT></SPAN><SPAN LANG="en-us"> <FONT FACE="Calibri">I would like to be able to easily set up 100+ test cases that are all test the same API, but with different</FONT></SPAN><SPAN LANG="en-us"> <FONT FACE="Calibri">URL parameter</FONT></SPAN><SPAN LANG="en-us"> <FONT FACE="Calibri">values and different expected results</FONT></SPAN><SPAN LANG="en-us"><FONT FACE="Calibri">.</FONT></SPAN><SPAN LANG="en-us"><FONT FACE="Calibri"></FONT></SPAN><SPAN LANG="en-us"> <FONT FACE="Calibri">I would</FONT></SPAN><SPAN LANG="en-us"> <FONT FACE="Calibri">to be able to flexibly configure these test case</FONT></SPAN><SPAN LANG="en-us"><FONT FACE="Calibri">s</FONT></SPAN><SPAN LANG="en-us"><FONT FACE="Calibri">, ideally as a</FONT></SPAN><SPAN LANG="en-us"> <FONT FACE="Calibri">text</FONT></SPAN><SPAN LANG="en-us"> <FONT FACE="Calibri">table or other simple</FONT></SPAN><SPAN LANG="en-us"><FONT FACE="Calibri">-to-create-and-parse data structure, and generate a report of pass/fail cases for each test. </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT FACE="Calibri">I would</FONT></SPAN><SPAN LANG="en-us"> <FONT FACE="Calibri">really like to use something like twill, but I</FONT></SPAN><SPAN LANG="en-us"><FONT FACE="Calibri">’</FONT></SPAN><SPAN LANG="en-us"><FONT FACE="Calibri">m getting stuck on 2 fronts:</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT FACE="Calibri">1. </FONT></SPAN><SPAN LANG="en-us"></SPAN><SPAN LANG="en-us"></SPAN><SPAN LANG="en-us"> <FONT FACE="Calibri">How to configure</FONT></SPAN><SPAN LANG="en-us"> <FONT FACE="Calibri">a single</FONT></SPAN><SPAN LANG="en-us"><FONT FACE="Calibri"> test case.</FONT></SPAN><SPAN LANG="en-us"><FONT FACE="Calibri"> The key here is that to check for valid results I need to unserialize the json returned result. Twill obviously doesn</FONT></SPAN><SPAN LANG="en-us"><FONT FACE="Calibri">’</FONT></SPAN><SPAN LANG="en-us"><FONT FACE="Calibri">t have this built in, so I assume I will have to</FONT></SPAN><SPAN LANG="en-us"><FONT FACE="Calibri"> call a twill script from python and then record its results for</FONT></SPAN><SPAN LANG="en-us"> <FONT FACE="Calibri">subsequent</FONT></SPAN><SPAN LANG="en-us"><FONT FACE="Calibri"> processing. Is that right? Should the calling code itself then be part of a unit testing framework?</FONT></SPAN><SPAN LANG="en-us"></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT FACE="Calibri">2. </FONT></SPAN><SPAN LANG="en-us"> <FONT FACE="Calibri">How to configure the series of test cases.</FONT></SPAN><SPAN LANG="en-us"><FONT FACE="Calibri"> I would like to avoid writing a separate test class / function for each case, and instead somehow iterate over the param/result pairs. I can</FONT></SPAN><SPAN LANG="en-us"><FONT FACE="Calibri">’</FONT></SPAN><SPAN LANG="en-us"><FONT FACE="Calibri">t figure out how to do this without having the entire suite abort after the first test failure. I would like to run all 100 tests, and have it report back which ones succeeded and which ones failed.</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"></SPAN><SPAN LANG="en-us"></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT FACE="Calibri">Are</FONT></SPAN><SPAN LANG="en-us"><FONT FACE="Calibri"> there option</FONT></SPAN><SPAN LANG="en-us"><FONT FACE="Calibri">s or configurations</FONT></SPAN><SPAN LANG="en-us"><FONT FACE="Calibri"> available that allow the flexibility of</FONT></SPAN><SPAN LANG="en-us"> <FONT FACE="Calibri">a simple</FONT></SPAN><SPAN LANG="en-us"><FONT FACE="Calibri"> configuration file</FONT></SPAN><SPAN LANG="en-us"> <FONT FACE="Calibri">/ data structure</FONT></SPAN><SPAN LANG="en-us"> <FONT FACE="Calibri">but</FONT></SPAN><SPAN LANG="en-us"><FONT FACE="Calibri"> that</FONT></SPAN><SPAN LANG="en-us"><FONT FACE="Calibri"></FONT></SPAN><SPAN LANG="en-us"> <FONT FACE="Calibri">will</FONT></SPAN><SPAN LANG="en-us"> <FONT FACE="Calibri">still run through all the tests as individuals?</FONT></SPAN><SPAN LANG="en-us"><FONT FACE="Calibri"> Or is this a mismatch for twill</FONT></SPAN><SPAN LANG="en-us"><FONT FACE="Calibri">’</FONT></SPAN><SPAN LANG="en-us"><FONT FACE="Calibri">s design?</FONT></SPAN><SPAN LANG="en-us"><FONT FACE="Calibri"></FONT></SPAN><SPAN LANG="en-us"> </SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT FACE="Calibri">FWIW, I</FONT></SPAN><SPAN LANG="en-us"><FONT FACE="Calibri">’</FONT></SPAN><SPAN LANG="en-us"><FONT FACE="Calibri">ve looked at funkload for this as well, but it isn</FONT></SPAN><SPAN LANG="en-us"><FONT FACE="Calibri">’</FONT></SPAN><SPAN LANG="en-us"><FONT FACE="Calibri">t clear how to handle the configuration issue above either, and unfortunately funkload</FONT></SPAN><SPAN LANG="en-us"><FONT FACE="Calibri">’</FONT></SPAN><SPAN LANG="en-us"><FONT FACE="Calibri">s nice reporting functionality doesn</FONT></SPAN><SPAN LANG="en-us"><FONT FACE="Calibri">’</FONT></SPAN><SPAN LANG="en-us"><FONT FACE="Calibri">t work for functional testing, only load testing</FONT></SPAN><SPAN LANG="en-us"><FONT FACE="Calibri">, so I</FONT></SPAN><SPAN LANG="en-us"><FONT FACE="Calibri">’</FONT></SPAN><SPAN LANG="en-us"><FONT FACE="Calibri">m not actually gaining that much</FONT></SPAN><SPAN LANG="en-us"><FONT FACE="Calibri">. I</FONT></SPAN><SPAN LANG="en-us"><FONT FACE="Calibri">’</FONT></SPAN><SPAN LANG="en-us"><FONT FACE="Calibri">m looking at other options right now</FONT></SPAN><SPAN LANG="en-us"><FONT FACE="Calibri">, with twill and nose on the shortlist. Any pointers would be greatly appreciated!</FONT></SPAN><SPAN LANG="en-us"></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT FACE="Calibri">Thank you in advance for your time.</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT FACE="Calibri">Thanks,</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT FACE="Calibri">Ramon</FONT></SPAN><SPAN LANG="en-us"></SPAN></P>
<p></p><p> This email message (and any attached document) contains information from Ingenuity Systems Inc. which may be considered confidential by Ingenuity, or which may be privileged or otherwise exempt from disclosure under law, and is for the sole use of the individual or entity to whom it is addressed. Any other dissemination, distribution or copying of this message is strictly prohibited. If you receive this message in error, please notify me and destroy the attached message (and all attached documents) immediately.</BODY>
</HTML>