¶ Data centre location and service level
The solution is offered in SaaS mode and hosted in two separate datacenters.
The first datacenter is called PA2 and is offered by the company Equinix in the Ile de France. The second is the Netcenter of the company SFR in the Auvergne-Rhône-Alpes region.
These data centres offer a very high level of security and are Tier 4 certified, thus providing an annual availability rate of 99.99% based on redundancy at the level of :
- Connectivity: Dual fibre penetration into the buildings from two separate telecom carrier routes; Rapidly installed cross-connects, including interconnecting cabling throughout the campus via a dark fibre network between all campus buildings.
- Power: Secure power supply. Two power supplies supported by inverters and generators for infinite autonomy at full capacity.
- Cooling system in N+1. Early warning smoke and water leakage detection system. Fire extinguishing system in accordance with local regulations, for maximum safety and minimum damage.
- As well as environmental provisions: Contractual commitment on service levels for air temperature and humidity according to ASHRAE 2011 recommendations
All data is therefore stored in France. All environments, servers and equipment are exchanged with data encryption for maximum security.
¶ The components of the solution
The solution is hosted in two Datacenters to ensure the highest level of availability of the infrastructure.
The main components of the solution hosted in the Datacenters are
¶ TSS IpTrade
The IP Trade TSS is a web-based database and management application that allows for system configuration and management while acting as a central repository for user profiles, enabling secure user authentication and system data storage. The TSS is also a gateway to other information systems and business applications that can be integrated into the trading room environment.
¶ TPO IpTrade
Each board is connected to the IP Trade media processing application called Turret Proxy Openline (TPO) and the PBX. The TPO acts as a standard unified communications network extension using Session Initialization Protocol (SIP) for connection and provides advanced interaction management functions for trading and command rooms.
¶ Cisco Telephony
Based on the Cisco Unified Communications platform, the Business Edition 6000 (BE 6000) brings together all the core communication options in a single platform. It offers calling, video, mobility, messaging, presence and contact centre functions. The Cisco Business Edition 6000 can connect up to 1000 employees.
¶ ASC recordings
ASC enregistre, analyse et évalue les interactions sur tous les supports, notamment la voix fixe, la voix mobile, le chat, la vidéo, l'écran, les réseaux sociaux et les SMS. L'enregistreur est certifié par IPTrade.
Le contenu des communications devient accessible, de sorte que les informations et tendances critiques sont révélées, fournissant en temps réel des informations commerciales et non commerciales pour une réaction immédiate de l'équipe. Les enregistrements et l'archivage sont cryptés et consultables pendant un minimum de 5 ans.
¶ OLT Services
OLT offers a unique tool at the confluence of Hypervision, business intelligence and process automation.
The objective is to enable you to achieve full centralised automation of monitoring functions for IT and Compliance teams to mitigate your companies' risk.
We also offer decision support for the business to improve operational results.
¶ Administration
The administration tools include the following:
- An integrated security information management solution combining a Directory Server, Kerberos manager, NTP server, DNS server, PKI certificate system, SSSD and other functions.
These tools are built on well-known components and standard protocols. The emphasis is on ease of management and automation of installation and configuration tasks. These tools are fully multi-master replicated for redundancy and scalability.
The entire infrastructure is based on LDAP authentication. All our applications are able to query the same LDAP directory and allow user authentication with a single identifier for all these services. Our directories support connection encryption mechanisms (SSL TLS).
Access rights to the various data in the directory and applications are finely managed thanks to Access Lists.
A "Zero-Touch Deployment" type system is also in place to facilitate the installation of equipment. The principle is to distribute the configurations directly when the systems are started up so that no manual action is required on the remote sites.
Thus, it is simply necessary to plug in the equipment and turn it on so that it can be configured autonomously.
¶ Session Border Controllers
SBCs are virtual edition audiocodes that are responsible for transmitting and receiving calls from the outside. They are installed in active/standby mode with a real time copy of the call status table. This allows the call session to be maintained in the event of a switch from active to passive SBC. In addition, the SBCs are in charge of call normalisation (number format and codecs).
¶ Virtualisation
We use Proxmox VE which is a complete platform for all-inclusive infrastructure virtualisation that tightly integrates KVM hypervisor and LXC containers, software-defined storage and networking capabilities on a single platform, and easily manages high-availability clusters and disaster recovery tools. Proxmox makes it easy to virtualise the most demanding Linux and Windows application workloads, and dynamically scale IT infrastructure and storage as needs change, to remain adaptable for future infrastructure growth.
¶ Functional architecture
The diagram below illustrates the components of the telephony solution
¶ TouchPro: Turret BT Trading
High performance touch panel. Each unit comes with 2 handsets, 1 speakerphone and a gooseneck microphone. These are the physical telephone terminals available to users.
¶ FlexPro
Turret software (running on Windows PC/tablet) with the same user interface as the TouchPro. Maximum 3 speaker channels as default configuration. Can be equipped with an XMA2 module and a peripheral kit (speaker, 2 handsets, microphone).
¶ Turret Proxy Open Line Server (TPO)
The BT Turret Proxy Open line (TPO) software is installed as a virtual application on a standard server and provides collaboration functions that cannot be offered by the PBX alone (call notification, sharing, etc.).
¶ Turret Support Server (TSS)
BT Trading administration application. This is the main administration portal for the trading environment. The TSS software is installed as a virtual application on standard servers. The TSS is an IIS (Internet Information Services) based web application, which provides interaction with a MySQL database. This database is used to store the configuration and preferences (profiles) of each turret user, and to manage the configuration of the TPOs. The TSS is also responsible for managing the software versions of the different components of the solution (TouchPro and TPO).
The TSS is also used for user authentication, report generation, as well as the integration of third party solutions (LDAP, Outlook...).
¶ ReplayBox
Interface to the recording system (ASC EVOIPNEO, Nice NTR, Red Box, Verint 15.2) to give users the ability to search and playback recorded calls from their deck.
¶ Cisco Unified Communications Manager iPBX
Unified communications platform for call set-up and transport. This application handles requests from BT endpoints (TPOs) for call control. Multiple CUCM nodes are used in a cluster to form a resilient call control mechanism. A dedicated CUCM 12 cluster is set up (2 nodes: 1 per DC).
¶ Voice recording
Voice recording system to meet compliance obligations. Active recording is used, with the decks (TouchPro or FlexPro) sending their voice streams directly to the recording systems. Dual streaming is used: the decks duplicate each recording stream and send them to 2 independent recording systems so as not to be impacted by the failure of one recording system or datacenter. Mixing is used for handsets (1 recording channel for both handsets) and speaker channels (3:1).
¶ Server
All components of the solution (TPO, TSS BT Trading, CUCM, registration, supervision) are installed as Proxmox virtual machines in the DC 1 / DC 2 environments.
¶ Network
A dedicated LAN is set up at the user sites to separate the OLT environment from our customers' network. The TouchPro panels are connected to this LAN. The LAN and WAN (for interconnection of sites) are provided by our customers.
¶ Description of the flows
¶ Making a call
When the board wants to make a call, the active TPO will communicate with the BE6K (Cisco CUCM IPBX) to evaluate if the call can go out, and then choose the routing table for the call (internal call or external call). The call is then sent to the SBC which will normalise the format of the number in +33 before sending it to the operator network. The TPO will launch a SIP call (UDP 5060 for signalling and UDP 1024/49151) to the BE6K (Cisco CUCM IPBX) which will then be transferred to the SBC.
1: The station launches its call (Signalling) which will go out through the active WAN access (Fibre by default) (UDP: 5080)
2: The signalling is carried to the IPBX (BE6K) through the front-end firewall. (UDP: 5080)
3: The stream reaches the IPBX and the IPBX will examine whether the call can go out, and to which equipment to send the call. The SIP signalling is then sent to the SBC. (UDP: 5080)
4: The IPBX will present the call to the SBC for exit via SIP signalling. (UDP: 5080)
5 : The call goes to the operator network (SIP flow between the Cisco BE6K IPBX and the VRRP address of the operator) and the RTP flow is thus set up between the station and the SBC (UDP Signalling: 5080 and RTP UDP 1024/49151)
¶ Receiving a call
When a call is presented to the infrastructure, the SBC will send the call to the Cisco BE6K IPBX which will examine its internal directory to find a correspondent. The signalling flows are SIP UDP5060 and RTP UDP 1024-49151.
1: The SBC receives the signalling from the operator network and presents the call to the IPBX (UDP: 5080) (The source address is the IP VRRP on the operator side, directed to the WAN address of the SBC)
2: The CUCM checks the call recipient and will send the call to the IP address of the extension to ring via a SIP signalling stream (UDP: 5080)
3: The signalling is sent to the extension through the front-end firewall (UDP: 5080)
4: The stream is routed to the telephone set (UDP: 5080)
5: The call rings the extension and the RTP flow is set up between the extension and the SBC (UDP Signalling: 5080 and UDP RTP 1024/49151)
¶ Call recording
When a call is made or received, a CTI link between the TSS and the board will trigger the start of the recording. This CTI link will send the call information (date, time, user name and phone numbers). Then, the CTI link will ask the telephone set to send a copy of its call flow to both recorders simultaneously. The signalling streams are sent in UDP5060 and RTP UDP 1024-49151. When a call is in progress, the extension will duplicate its RTP stream to both recorders.
1: The station sends a copy of the RTP flow to the two recorders as well as the call information to the CTI interface of the recorders, the flow exits through the local router (the default is fibre) UDP: 5080
2: The stream is routed to the two recorders UDP: 5080
3: The audio stream is presented to the recorder in the first datacenter UDP: 5080
4: The audio stream is presented to the recorder of the second datacenter UDP: 5080
¶ User's login
When a user authenticates to the board, it will query its TSS via FTPS to download the user's profile. Authentication is delegated to an LDAP directory.
The flows are as follows:
FTPes: TCP 20/21 (From the board to the TSS) to retrieve updates and configuration from the boards
LDAP-S: TCP 636 (From the board to the TSS, then from the TSS to the LDAP servers) to authenticate users
The authentication kinematics are as follows:
1: The board will try to contact the Load Balancer for LDAP requests through its local router
2: The flow is routed in the core network to our datacenter infrastructure
3: The flow arrives at the HA-PROXY load balancer which will send the flows to one of the available LDAP servers
Once the authentication is complete, the FTP flows go to the board with the board configuration and updates if necessary
¶ Playback from the turntable
When a user wants to replay the deck, the deck will query the recording server in Paris or Lyon.
The streams are as follows:
HTTP Replay: TCP 80 (From the deck to the recorder) to retrieve the replay files
1: The deck will contact the replay server by going out through its local gateway
2: The stream is routed to the infrastructure
3: The request is directed by the playback interface to the station
4: The request is handled by the recorder which will open a TCP80 session to the deck
¶ Firewall and flows
¶ Communications between IPTRADE components
|
From |
To |
Protocol |
Port number |
Usage |
|
PBX |
Turret |
UDP/TCP |
5060 |
SIP - Telephony |
|
TPO |
||||
|
TPO |
UDP |
11000-13000 |
Voice RTP Ports |
|
|
Gateway |
||||
|
TPO |
UDP |
31000-31100 |
Video RTP Ports |
|
|
Gateway |
||||
|
PBX |
TPO |
UDP/TCP |
5060-5860 |
SIP - Telephony (one per PBX device) |
|
Turret |
UDP/TCP |
5060 |
SIP - Telephony |
|
|
Turret |
UDP |
10000-30000 |
Voice and Video RTP Ports |
|
|
Gateway |
||||
|
Turret |
TSS |
TCP |
21 |
FTPES |
|
Turret |
TCP |
443 |
Secured communication between turret and TSS (HTTPS) |
|
|
Turret |
UDP |
514 |
Syslog |
|
|
Turret |
TCP |
1024-65535 |
FTP passive data ports |
|
|
Turret |
DHCP Server |
TCP/UDP |
67-68 |
DHCP |
|
Turret |
Gateway |
UDP |
This range is a parameter from the Gateway |
Voice and Video RTP Ports |
|
TPO |
||||
|
Turret |
PBX |
UDP/TCP |
5060 |
SIP - Telephony |
|
TPO |
||||
|
TSS |
UDP |
69 514-65000 |
SIP device fetch : •69 for the first TFTP request •514-65000 for the rest of the TFTP communication |
|
|
Turret |
LDAPS |
TCP |
636 |
Communication between LDAPS and Turret |
|
TSS |
||||
|
TSS |
SQLServer |
TCP |
1433 |
Communication between TSS & SQLServer (also used for replication) |
|
TSS |
MySQL |
TCP |
3306-3307 |
Communication between TSS & MySQL (also used for replication) |
|
Turret |
ASC ReplayBox |
TCP |
8024 |
Communication between Turret and the replay recorder server for the replaybox |
|
Turret |
IPTrade ReplayBox |
TCP |
80 |
Communication between Turret and the replay recorder server for the replaybox |
|
Turret |
DNS Server |
UDP |
53 |
DNS |
¶ Communications to the turntables
|
ReplayBox |
Port number |
|
ReplayBox ASCNeo |
80 |
¶ Flow diagram with a physical board
¶ Flow diagram with a board on VPN
¶ Deck flow matrix
|
Protocol |
Port number |
Usage |
Default state |
Related Configuration keys |
|
TCP |
80 |
Web Administration |
Enabled |
application.boot.startserver |
|
UDP |
161 |
SNMP |
Enabled |
application.global.snmp.enable |
|
application.global.snmp.read.community |
||||
|
application.global.snmp.trap.community |
||||
|
TCP |
2002 |
Remote Packet Capture |
Disabled |
application.global.rpcapd.enable |
|
profile.setting.debuginfobutton.alwaysavailable |
||||
|
TCP |
2305 |
XMA Switch Administration |
Disabled |
Can be enabled/disabled only when turret has XMA via following setting: |
|
application.xma.telnetaccess |
||||
|
TCP |
4250-4251 |
OLDCB - Internal mixing matrix |
Enabled (No XMA) |
Required for mixed recording without XMA |
|
Disabled (XMA) |
||||
|
TCP/UDP |
5060 |
SIP - Telephony |
Enabled |
application.sip.connection.port |
|
TLS |
5061 |
SIP - Telephony |
Disabled |
application.sip.tlsconnection.port |
|
TCP |
80 |
SIP - Telephony over WebSocket |
Disabled |
application.sip.websocket.listening.port |
|
TLS |
443 |
SIP - Telephony over WebSocket Secure |
Disabled |
application.sip.websocketsecure.listening.port |
|
UDP |
9876 |
ITMH |
Disabled |
profile.setting.broadcasting.enabled |
|
profile.setting.broadcasting.mcast |
||||
|
UDP |
11000-13000 |
Voice RTP Ports |
Enabled |
application.mm.rtplowerport |
|
application.mm.rtpupperport |
||||
|
UDP (XMA2 only) |
10240-12286 (default) or |
Voice RTP Ports |
Enabled |
application.xma.trafficin.portef |
|
11264-12286 or |
application.xma.trafficin.netmask |
|||
|
16384-20478 or |
application.xma.trafficin.portrange |
|||
|
18432-20478 or |
application.mm.rtplowerport |
|||
|
32768-36862 or |
application.mm.rtpupperport |
|||
|
34816-36862 |
||||
|
UDP |
31000-31100 |
Video RTP Ports |
Enabled |
application.mm.vie.rtplowerport |
|
application.mm.vie.rtpupperport |
¶ Resilience mechanisms
| Equipement | Redondance |
| SBC |
Active/Passive (Automatic switchover) The switchover takes place when the standby SBC can no longer reach the nominal SBC on its H-A interface. Disasters are detected when the SBCs fail to respond to ping. |
| TSS |
Active / Active (No failover) Databases are replicated in real time. Disasters are detected when the web interface is down as well as the OS side metrics. |
| BE6K |
Active / passive (Automatic switchover) The board will register to the new CUCM after a 30 second timeout Databases are replicated in real time. Disasters are detected when the WEB interface is down and the metrics on the OS side. |
| OLT |
Active / Active (No failover) The containers used by OLT are based on a disk space with block replication (Gluster). Disasters are detected when the WEB interfaces are unreachable. |
| TPO |
Active / Active (No toggle) All 4 TPOs are active at the same time. Disasters are detected when the TPOs no longer issue metrics on the OS side. |
| ADMIN |
Active / Active (No toggle) The containers used by the Admin layer are based on a disk space with block replication (Gluster). Disasters are detected when the WEB interfaces are unreachable. |
| ASC |
Active / Active (No toggle) 2N capture and replication of data between the two loggers. Losses are detected when the WEB interface is down as well as the metrics on the OS side. |
| SIP telephony trunk access |
Active / Active (No toggle) The trunks are permanently accessible from both SBCs (Inbound and Outbound). Disasters are detected via SNMP and SYSLOG from the SBCs. |
¶ Impacts in the event of an outage or switchover
|
Type of failure |
Impact |
Solution |
Duration of unavailability of calls |
|
WAN link failure at a customer site |
Total disconnection of the service |
Automatic switchover to SDSL backup |
Break of a few seconds |
|
CUCM IPBX shutdown |
Automatic switchover to SDSL backup. |
Registration on the backup CUCM * |
Less than a minute |
|
Cutting a TSS |
Inability for users to view call logs and edit their configuration |
Automatic switchover to the second active TSS |
None |
|
Cutting a TPO |
Inability to handle calls |
Automatic switchover to the second active TPO |
Less than a minute |
|
Disconnection of a recorder |
None (Continuity of recording on the second recorder) |
Continuity of recording on the second recorder |
None |
|
Cutting the administration base and / or Open Lake |
Inability to authenticate |
Automatic switchover to the standby base (Active / Active) |
None |
|
Nominal router outage |
Network unavailable for a few seconds |
Automatic switchover to the backup router |
Break of a few seconds |
|
Backup router outage |
None |
None |
None |
|
Coupure switch client 1 |
Network unavailable for a few seconds |
Automatic switchover to the backup switch |
Break of a few seconds |
|
Coupure switch client 2 |
Network unavailable for a few seconds |
Automatic switchover to the backup switch |
Break of a few seconds |