When it comes to telephones, everybody has certain expectations.
Nobody wants a phone that fails occasionally. And most service providers have a model of phone that they trust; nobody wants to try out a new SIP phone and learn that it fails in an obscure, but important feature.
For example: Have you seen the new Yealinks? Or what about the improved Grandstreams? How do you ensure the phone is going to do everything you need it to do -- like those unusual things a receptionist does, that a technician or engineer never does?
Below you can find a set of 152 Test Criteria (TC's) I've used to test new models of SIP phones. If you develop test procedures to confirm each of these criteria are met, you'll have a good idea that the SIP phone is going to do all the PBX functions for a SIP phone.
Using these criteria requires you to have a general understanding of Service Provider VoIP. To develop a test procedure to confirm TC1, "Proper Registration via SBC." The value of this list is to help you remember things you might have forgotten otherwise, and expedite testing procedures.
Mark's 152 Test Criteria for SIP Phones
TC1 : Proper Registration via SBC
TC2 : Symmetric signaling (SIP to and from the same UDP port) and media (RTP to and from the same port)
TC3 : PSTN to SIP call, calling party disconnects first
TC4 : PSTN to SIP call, called party disconnects first
TC5 : SIP to PSTN call, calling party disconnects first
TC6 : PSTN to SIP call, called party disconnects first
TC7 : SIP to SIP call for each supported model or software version (e.g., Aastra 57i/2.2.0 calling to PolyCom SPIP 601)
TC8 : Calling-Party Name and Number display for "on-net" VoIP-CPE-to-VoIP-CPE calls
TC9 : Calling-Party Name and Number display for calls received from each PSTN carrier
TC96 : Called Party name displays on intra-group calls and calls to group-call-park
TC29 : Auto-answer for click-to-dial calls (i.e., supports SIP INVITE with alert to make calling phone automatically pick up)
TC39 : Calls sent to the DUT but not answered
TC40 : Calls placed by DUT but not answered
TC41 : DUT installed behind firewall with NAT
TC42 : DUT installed behind firewall with no NAT
TC43 : SIP over TCP (for devices that support it)
TC103 : Call Forward Always using key on phone and signaling on phone
TC51 : Call Forward Always using key on phone, synchronized with BroadWorks/Metaswitch/Feature Server settings
TC104 : Do-Not-Disturb using key on phone and signaling on phone
TC52 : Do-Not-Disturb using key on phone, synchronized with BroadWorks/Metaswitch/Feature Server settings
TC53 : DUT can be configured match Service Provider's standard digit map (aka "dial plan")
TC106 : DUT can dial calls the same whether the phone is off hook or on hook
TC84 : DUT re-registers a 5-15 seconds before the old registration expires.
TC10 : Call hold and retrieve using hold button or soft key on DUT
TC11 : Remote-party call hold and retrieve (i.e., the remote end of the conversation puts the call on hold, not the DUT)
TC12 : Three-way calling
TC13 : Three-way call disposition when DUT disconnects from call
TC14 : Blind transfer
TC15 : Consultative transfer
TC16 : Shared Call Appearance (SCA)
TC17 : Busy Lamp Field (BLF)
TC18 : Multiple lines configured on one telephone
TC19 : Proper NTP synchronization of clock display
TC20 : Long-call-duration calls
TC21 : Call Forward Always using key on phone
TC22 : Do-Not-Disturb using key on phone
TC23 : Multiple buttons configured for the same line
TC24 : DTMF transfer to Media Server, Conferencing Server, Auto Attendant server, etc.
TC25 : DTMF transfer to PSTN via PSTN gateway(s)
TC57 : PLAR / Automatic Ring-Down: DUT can be configured to automatically dial a number when the handset is lifted
TC67 : DUT should support up to 6 inbound calls stacked up (ringing inbound to phone) and function properly
TC68 : When you have lots of inbound calls stacked up, you can put a call on hold and park the most-recently-active call
TC69 : When call parked by the DUT reverts to the DUT phone, the DUT phone can retrieve the call properly
TC111 : When DUT transfers a call, and the Call Transfer Recall timer reverts the call back to DUT, it can retrieve the call properly
TC70 : DUT can call into a 10-line paging group and automatically announce to those lines
TC71 : DUT can be a member of a 10-line paging group, so that when the group is called it automatically goes on speaker and announces what the caller says
TC72 : (Push-to-Talk) Hoot-and-Holler call outbound: DUT can call to another phone and have it go off hook on speakerphone
TC73 : (Push-to-Talk) Hoot-and-Holler call inbound: DUT can receive an inbound call and have it go off hook on speakerphone
TC110 : Auto-answer "Push-to-talk" calls and sets up two-way conversation
TC85 : DUT supports Broadworks/Metaswitch N-Way conferencing (Low-priority test).
TC86 : DUT can put a call on hold and retrieve it 5 times, and have two-way audio each time the call is un-held.
TC87 : DUT can be on a call where the remote party puts the call on hold and retrieves is 5 times, and have two-way audio each time the call is un-held.
TC112 : DUT works properly with BroadWorks/Metaswitch Group Call Park
TC88 : DUT operates as any party in any call transfer scenario
TC88.1 : Blind transfer, original recipient is facilitator
TC88.1.a : Blind transfer, original recipient is facilitator, DUT is originator
TC88.1.b : Blind transfer, original recipient is facilitator, DUT is original recipient
TC88.1.c : Blind transfer, original recipient is facilitator, DUT is final recipient
TC88.2 : Blind transfer, originator is facilitator
TC88.2.a : Blind transfer, originator is facilitator, DUT is originator
TC88.2.b : Blind transfer, originator is facilitator, DUT is original recipient
TC88.2.c : Blind transfer, originator is facilitator, DUT is final recipient
TC88.3 : Blind transfer before call answer, original recipient is facilitator
TC88.3.a : Blind transfer before call answer, original recipient is facilitator, DUT is originator
TC88.3.b : Blind transfer before call answer, original recipient is facilitator, DUT is original recipient
TC88.3.c : Blind transfer before call answer, original recipient is facilitator, DUT is final recipient
TC88.4 : Blind transfer before call answer, originator is facilitator
TC88.4.a : Blind transfer before call answer, originator is facilitator, DUT is originator
TC88.4.b : Blind transfer before call answer, originator is facilitator, DUT is original recipient
TC88.4.c : Blind transfer before call answer, originator is facilitator, DUT is final recipient
TC88.5 : Attended transfer, original recipient is facilitator
TC88.5.a : Attended transfer, original recipient is facilitator, DUT is originator
TC88.5.b : Attended transfer, original recipient is facilitator, DUT is original recipient
TC88.5.c : Attended transfer, original recipient is facilitator, DUT is final recipient
TC88.6 : Attended transfer, originator is facilitator
TC88.6.a : Attended transfer, originator is facilitator, DUT is originator
TC88.6.b : Attended transfer, originator is facilitator, DUT is original recipient
TC88.6.c : Attended transfer, originator is facilitator, DUT is final recipient
TC88.7 : Busy-Failed Blind transfer, original recipient is facilitator
TC88.7.a : Busy-Failed Blind transfer, original recipient is facilitator, DUT is originator
TC88.7.b : Busy-Failed Blind transfer, original recipient is facilitator, DUT is original recipient
TC88.7.c : Busy-Failed Blind transfer, original recipient is facilitator, DUT is final recipient
TC88.8 : Busy-Failed Blind transfer, originator is facilitator
TC88.8.a : Busy-Failed Blind transfer, originator is facilitator, DUT is originator
TC88.8.b : Busy-Failed Blind transfer, originator is facilitator, DUT is original recipient
TC88.8.c : Busy-Failed Blind transfer, originator is facilitator, DUT is final recipient
TC88.9 : No-Answer-Failed Blind transfer, original recipient is facilitator
TC88.9.a : No-Answer-Failed Blind transfer, original recipient is facilitator, DUT is originator
TC88.9.b : No-Answer-Failed Blind transfer, original recipient is facilitator, DUT is original recipient
TC88.9.c : No-Answer-Failed Blind transfer, original recipient is facilitator, DUT is final recipient
TC88.10 : No-Answer-Failed Blind transfer, originator is facilitator, DUT is originator
TC88.10.a : No-Answer-Failed Blind transfer, originator is facilitator, DUT is originator
TC88.10.b : No-Answer-Failed Blind transfer, originator is facilitator, DUT is original recipient
TC88.10.c : No-Answer-Failed Blind transfer, originator is facilitator, DUT is final recipient
TC44 : DUT downloads via FTP/TFTP/HTTP/HTTPS and ability to detect and download new configurations
TC49 : Request IP address via DHCP upon phone startup
TC50 : Request IP address after lease expiration or after loss of Ethernet connectivity without power-cycling phone
TC46 : Detect voice and VLAN tags automatically (e.g., using CDP or 802.1AB/LLDP)
TC47 : 802.1q VLAN tagging on "LAN" port
TC48 : Function as Ethernet switch or hub for PC port. The DUT should switch data-VLAN traffic to the PC port, removing the VLAN tag. It should also switch frames received on the PC port to the LAN port, adding the appropriate VLAN tag.
TC54 : When using PoE, DUT should request a reasonable amount of power
TC56 : The switch in the phone should pass through untagged frame (i.e., any frame with no 802.1q vlan tag) from between the PC port and the LAN port to support the PC/access VLAN being untagged while the voice VLAN is tagged.
TC59 : DUT can locate the SIP server using SRV records.
TC60 : Locate the SIP server using A-record lookups for an outbound proxy.
TC64 : When using a wireless handset, it should work in the presence of moderate or heavy 802.11 traffic
TC74 : DUT can be rebooted remotely from the Reset function in BroadWorks/Metaswitch
TC75 : DUT creates logs of call activities and events
TC76 : DUT logs call statistics (packet loss, jitter, codec, etc.)
TC77 : DUT can upload its log to a server using FTP
TC78 : DUT with built-in switch can support wire-rate 100 Mbps for a PC plugged in behind the phone
TC79 : DUT with built-in switch works OK if device behind PC sends wake-on-LAN packets when it goes to sleep or wakes from sleep
TC80 : DUT works when connected to a network with 10% broadcast or multicast traffic loads
TC81 : DUT does not have a significantly-increased CPU load level in presence of 10% broadcast or multicast traffic loads (low-priority test: do this last)
TC82 : DUT works when connected to a standard customer-premise network switch configuration
TC89 : DUT can automatically detect when new firmware is available (by polling the server to check for new software), then selectively download and upgrade itself only when new software is available.
TC90 : DUT downloads new firmware using authenticated FTP.
TC93 : DUT can download new configurations and configurations via FTP using the username and password specified with DHCP response Option 66
TC94 : DUT can download new configurations via HTTPS using the username and password specified with DHCP response Option 66
TC105 : A brand-new-out-of-the-box DUT can be upgraded automatically, without manually logging into the DUT and triggering the upgrade
TC108 : DUT should work with slow FTP servers
TC107 : If DUT is unable to connect to the Configuration Server (FTP/HTTP/etc. server), it should use the configuration it last downloaded.
TC26 : RFC2833 DTMF send
TC27 : G.729ab and G.711u codec support
TC28 : 20 ms packetization time
TC30 : Ringback on outbound calls to PSTN
TC31 : Ringback on outbound calls to all other supported SIP devices
TC32 : Early media playback for calls to PSTN (e.g., for calls sent to announcement by PSTN but never connected)
TC33 : Early media playback for local calls (e.g., for calls to Sequential Ring in BroadWorks or Custom Ringback in other platforms)
TC34 : Proper handset, speakerphone, and external-headset operation
TC62 : TOS or DSCP value can be configured for signaling and for RTP.
TC63 : DUT generates RTCP for all RTP sessions.
TC95 : After SDP offer/answer, DUT uses its own preferred codec rather than the most-preferred codec in the SDP answer.
TC97 : DUT that supports G.729 can be configured to disable silence suppression
TC109 : DUT should support Multicast Paging that can be used with BroadWorks/Metaswitch
TC91 : DUT audio quality in both directions is OK in the presence of moderate to extreme electromagnetic interference (EMI)
TC35 : Acoustic echo cancellation or mitigation on handset and speakerphone
TC36 : Gain control for handset and speakerphone (to reduce talker loudness)
TC37 : Volume control to change loudness of remote party
TC38 : Proper and Working mute and un-mute operation
TC102 : DUT will cancel out far-end echo so the user of the DUT doesn't hear himself.
TC45 : Change call to use external headset
TC55 : If there are more softkey options available than there are softkeys, then DUT should show a "More..." button to get to the other softkey options.
TC58 : DUT can access and use vendor's demo XML applications via the Internet when the URL is specified as a hostname and not an IP address.
TC61 : Use one default Services Application on phones for which no device-specific configuration has been generated, and a different default Services Application on phones for which a device configuration has been generated.
TC65 : If DUT supports a sidecar, the buttons can be assigned to new lines or programmable as softkey actions, including dialing BroadWorks/Metaswitch feature codes
TC66 : If DUT supports a sidecar, the buttons behave like buttons built-in to the phone.
TC83 : If DUT works with two sidecars, any button on any sidecar should be assignable to line registrations or programmable hotkeys on the sidecars.
TC92 : Softkeys should be programmable to access a URL, speed dial a number or Broadworks/Metaswitch feature code.
TC96 : DUT should be configurable to strip leading digits in the dial plan.
TC99 : DUT can dial feature codes that begin with a pound character; e.g., "#58"
TC100 : DUT can configure BroadWorks/Metaswitch call park and call-retrieve to softkeys
TC101 : DUT can be configured to disable the Do-Not-Disturb button on the phone
Beyond the 152
Of course, some platforms are going to offer more features. So be on the lookout for:
- Video Conferencing
- Video Calling and Video Voicemail
- Call Center Login/Logout
- User-Activated Centralized Call Recording (e.g., SIPREC activation)
- Phone-Based Call Recording