Mu Takes VoIP Testing on the Road (Mu Dynamics Blog)
by Thomas Maufer on 18 April 2008 - 02:52:48 PM
Mu Takes VoIP Testing on the Road
Mu showed it's newest VoIP robustness and service assurance tools in 3 very public locations last week: cisco's Toolapalooza, Software Test & Performance and SIPit-22. This blog posting focuses on SIPit-22.
SIPit
is the premier Voice over IP (VoIP) and SIP interoperability "plugfest" bringing together
engineers from top vendors and service providers to try to make sure
their VoIP and SIP implementations work all the time, every time. New
VoIP features are generally more fragile than older applications that
are better understood. The value of events like SIPit is in finding
specific cases where things don't work (actually, finding things that
*do* work is equally valuable, and certainly gives an emotional
boost!). The participants are actively making their VoIP code more
robust, able to handle a wider variety of exception conditions.
In some
cases, the exceptions simply arise from getting your code to talk to
code you didn't write. That's a classic interoperability test. There
are also "torture chambers" here that test various intentionally
created challenges, like loops of proxies to test loop detection code,
including the ability to process large numbers of Via: headers. The
driving reason for participating in SIPit is the ability to proactively
(and secretly) test against a large number of other vendors, in a wide
variety of scenarios, to eliminate bugs that would otherwise remain
unseen until they surface in multi-vendor real-world deployments. When
products or features get that far, into real or nearly-real network
deployments, there is a substantial chance that carrier revenue will be
at risk due to VoIP service downtime.
That's
the real benefit: When specific things don't work, and are found here
at SIPit, participants are improving product quality which helps avoid
downtime in real deployments. Downtime costs money, and organized
testing is one of the only formal ways to manage or reduce downtime.
However, SIPit doesn't happen nearly often enough. What if a vendor
could expose its products to a "perfect storm" of unexpected traffic
every time they created a new build or were verifying newly patched
code against a regression baseline? Or what if carriers could do the
same kind of testing every time they were considering
a purchase, upgrade/patch or configuration change? Vendors who had
access to this testing could make it part of their secure development
life cycle and could test or spot-check their nightly or weekly builds
for regressions at will and as part of the release process. Vendors are
by far the dominant faction here at SIPit, but there are a few carriers
here are clearly looking to next-generation IP services that they plan
to roll out in the medium term, and are working closely with their
chosen vendors.
What vendors and carriers both
need is something like "SIPit-in-a-box" -- a service assurance solution applicable to on-demand use
throughout their respective life cycles. Mu's solution, the Mu-4000, is
able to throw a variety of attack types, including mutations (of
various VoIP protocols, e.g., H.323, SIP, H.248, MGCP, RTP/RTCP, etc.)
as well as denial-of-service attacks, both of which stress the control,
management, and data planes of typical VoIP deployments. Moreover, to
keep pace with the rapidly evolving world of SIP, Mu allows vendors to
configure custom headers and payloads that the Mu-4000 uses to create
custom-tailored mutations based on real, next-generation features, so
the test cases can be extremely specific to the features, even
bleeding-edge features that haven't been widely deployed yet.
Vendors are not the only beneficiaries
of service assurance testing: Carriers benefit heavily from VoIP
service assurance throughout their deployment
life cycle. They can test before they purchase, then they can create a
baseline against which future configuration changes or patches/upgrades
are tested, and evolve that baseline as the new testing is completed,
certifying new versions as production-ready. Eventually they build a
regression suite of their own and make sure that their vendors'
products can all play in the carrier's multi-vendor environment as it
evolves with the addition of new services that leverage the same core
infrastructure.
Carriers are uniquely able to do this kind of testing:
Practically speaking, vendors simply can't do in-house QA testing in a
multi-vendor environment. Vendors can, at best, only test against their
own equipment (with the exception of events like SIPit). Carriers will
put the products into configurations and situations that are unique to
the carrier's network. In the complex and highly competitive world
where new services mean new revenue streams, carriers are motivated to
roll out new products and new features quickly -- before they have time
to be thoroughly baked at a series of events such as SIPit. Service
assurance testing takes on new urgency in this highly competitive
environment.
SIPit offers Mu and other
participants a great event with lots of benefits to developers at
product vendors as well as engineers at carriers designing
next-generation networks. Another benefit of these events is that
occasionally participants discover that no matter how hard the
engineers try, they can't correctly interpret the spec. This is rare,
but when bugs are found in the spec it provides very useful real-world
feedback to the folks that are writing the standards.
Mu is participating to make sure that our extensive industry-leading
VoIP/SIP implementation talks to the widest variety of other
implementations by getting the requisite exposure to a much larger set
of targets than we can possibly fit in our lab. Also, when issues are
found, directly talking to other engineers helps converge on a fix, frequently in real time. Exposure to real implementations in realistic deployments
benefits all participants, and by actively participating in such
events, Mu is able to increase the quality of its solution, and
preserve its leadership position in the service assurance testing
market. SIPit-22 is now over, but we're already planning for SIPit-23 in Lannion, France.
Comments:
Write a comment
- Required fields are marked with *.
|
