1.
Cluster Drive Test Tools Setup LTE drive test needs to use attenuator for simulation indoor coverage. As the figure below shown, UE connect external antenna via attenuators, and putting external mini antenna and GPS on top roof of test vehicle. External antenna gain should offset the gap with total cable and connector loss.
Figure 2.1
Data Testing Illustration
Figure 2 .2
VoLTE Testing Illustration
List of tools for cluster test as following table
Items
Name of tools
Description
1
TEMS
DT software
2
TEMS Discovery
DT data analysis software
3
Test terminal
UE of cat3 ,cat4,cat6
4
SIM card
Speed unlimited.
5
GPS
USB GPS
6
Attenuator
Support four attenuated values, including 25db/20db/15db/10db
7
Power converter
Power converter (DC12V to AC220V) during drive test.
8
Notebook
Support TEMS softwares and other softwares like MapInfo etc.
9
Digital Map
TAB format, supporting TEMS software geographical display.
10
Vehicle
Have enough workspace with cigarette lighter or battery power supply.
11
Antenna
External Omni-antenna, 0dbi or cable and connector loss offset the gain.
12
Isolation box
Isolate external signal.
13
MOS equipment
Support LTE Voice MOS value (ascom, etc.)
The Different clutter attenuator configurations in cities are as shown in the following table 1)
2)
JKT, BGR & SBY set as following loss value Clutter
Attenuator
Dense Urban
25 dB
Urban
20 dB
Sub Urban
15 dB
Rural
10 dB
Other cities set as following loss value Clutter
Attenuator
Dense Urban
20 dB
Urban
15 dB
Sub Urban
10 dB
Rural
10 dB
2.
Cluster and Golden Cluster KPIs
2.1
Cluster KPIs
Round 2 Round 3
Short Call and Ping test DL Long Call UL Long Call
Round 4
VoLTE
Round 1
RRC Connection Request Success Rate, ERAB drop rate (or drop call rate), Packet Loss Handover, Handover Interruption time, Downlink Throughput Uplink Throughput VoLTE Setup Success Rate, VoLTE Dropout Rate, VoLTE Voice – Mean Opinion Score, VoLTE call setup time
2.1.1
Mobility KPIs
RRC Connection Request Success Rate 1. Test Procedures 1) After the test set up ready. 2) Start download 1 file which is 1G through internal FTP server for 60 seconds. 3) Detach terminal for 10 seconds. 4) Drive the vehicle and repeat step 2 and 3, till finish the test route for at least one time.
2. Signaling Flow chart
3) KPI Formula RRC Connection Request Success Rate= Total number of RRC Connection Request Success / Total number of RRC Connection Request *100%
ERAB drop rate (or drop call rate) 1. Test Procedures 1) After the test set up ready. 2) Start download 1 file which is 1G through internal FTP server for 60 seconds. 3) Detach terminal for 10 seconds. 4) Drive the vehicle and repeat step 2 and 3, till finish the test route for at least one time.
2. Signaling Flow chart
3. KPI Formula Main formula: ERAB drop rate (or drop call rate) = Number of drop calls * 100%/Total Number of calls For measurement, reflect to below formula: ERAB
drop
rate
(or
drop
call
rate)
=
Total
number
(RRCConnectionReestablishmentRequest-RRConnectionReestablishmentcomplete) *100% / Total number of Activate default EPS bearer context accept
Packet Loss 1. Test Procedures 1) After the test set up ready. 2) Ping from laptop to internal FTP server with 32 bytes.
of
3) Drive the vehicle, repeat step 2, and finish the test route for at least one time.
2. Formula Packets Loss= quantity of ping time out/ quantity of ping request * 100%
Handover Success Rate 1. Test Procedures 1) After the test set up ready. 2) Start download 10 files each of which is 1G simultaneously through internal FTP or UDP server. 3) Keep the downloading. If the downloading process is interrupted, like laptop hang up, stop the vehicles. Record the time. Then set up the connection and then go on with the download throughput test. The data during interruption is not effective. 4) Drive the vehicle to finish the pre-defined test route for at least one round.
2. Signaling Flow chart
3) Formula Handover Success Rate = total of successful intra frequency handover operations / quantity of intra frequency handover requests * 100%
Handover Interruption time 1. Test Procedures 1) After the test set up ready. 2) Start download 10 files each of which is 1G simultaneously through internal FTP or UDP server. 3) Keep the downloading. If the downloading process is interrupted, like laptop hang up, stop the vehicles. Record the time. Then set up the connection and then go on with the download throughput test. The data during interruption is not effective. 4) Drive the vehicle to finish the pre-defined test route for at least one round.
2. Signaling Flow chart
The latency is from 5(RRC Connection Reconfiguration) to 8(Random Access Response).
3. Formula Handover Interruption time Control Plane = total delay of all successful handover operations / quantity of successful handover operation. In case of abnormal situation (like fading due to rain, transmission latency from eNB to MME is more than 2ms, etc.) observed in transmission system, corrective action shall be done by ZTE and test will be re-offered to SF.
Downlink Throughput 1. Test Procedures 1) After the test set up ready. 2) Start download 10 files each of which is 1G simultaneously through internal FTP or UDP server. 3) Keep the downloading. If the downloading process is interrupted, like laptop hang up, stop the vehicles. Record the time. Then set up the connection and continue to the download throughput test. The data during interruption is not effective. 4) Drive the vehicle to finish the pre-defined test route for at least one round.
2. Formula DL User Throughput (50th Percentile) = total DL PDCP throughput /quantity of valid samples (exclude call drop area) DL User Throughput (5th Percentile) = 5% of DL PDCP throughput CDF statistics (exclude call drop area) DL User Throughput (95th Percentile) = 95% of DL PDCP throughput CDF statistics (exclude call drop area)
DL Throughput(except for DL peak throughput) is collected through normalization (Only one data point within a certain distance should be calculated, one data point every 10m. If there are several measurements within that distance the samples shall be averaged).
If no vendor can achieve the target peak value un der 25% loading then SF w ill review the results and may consider alternate method to test peak throughput Radio capability.
Uplink Throughput 1. Test Procedures 1) After the test set up ready. 2) Start upload 10 files each of which is 1G simultaneously through internal FTP or UDP server. 3) Keep the uploading. If the uploading process is interrupted, like laptop hang up, stop the vehicles. Record the time. Then set up the connection and continue to the upload throughput test. The data during interruption is not effective.
4) Drive the vehicle to finish the pre-defined test route for at least one round.
2. Formula UL User Throughput (50th Percentile) = total UL PDCP throughput /quantity of valid samples (exclude call drop area) UL User Throughput (5th Percentile) = 5% of UL PDCP throughput CDF statistics (exclude call drop area) UL User Throughput (95th Percentile) = 95% of UL PDCP throughput CDF statistics (exclude call drop area)
UL Throughput(except for UL peak throughput) is collected through normalization (Only one data point within a certain distance should be calculated, one data point every 10m. If there are several measurements within that distance the samples shall be averaged).
If no vendor can achieve the target peak value un der 25% loading then SF will review the results and may consider alternate method to test peak throughput Radio capability.
2.1.2
VoLTE KPI
VoLTE Setup Success Rate 1. Test Procedures 1) After the test setup is ready. 2) Use the script to make a VoLTE call and then hold the call connection for 30 seconds. 3) Use the script to hang up the terminal and wait for 15 seconds. 4) Drive the vehicle, repeat 2 to 3, and finish the pre-defined test route for at least one time.
2. Signaling Flow chart
3. Formula VoLTE call setup success rate = Quantity of completed VoLTE calls / total of VoLTE call connection requests*100%
VoLTE Dropout Rate 1. Test Procedures 1) After the test setup is ready. 2) Use the script to make a VoLTE call and then hold the call connection for 30 seconds. 3) Use the script to hang up the terminal and wait for 15 seconds. 4) Drive the vehicle, repeat step 2 to 3, and finish the pre-defined test route for at least one time.
2. Signaling Flow chart
3. Formula VoLTE call drop rate = quantity of VoLTE call drops /quantity of completed VoLTE calls *100%
VoLTE Voice
Mean Opinion Score
–
1. Test Procedures 1) Tool setup for MOS follow figure 2.2 VoLTE Testing Illustration. 2) Use the script to make a VoLTE call and then hold the call connection for 30 seconds. 3) Use the script to hang up the terminal and wait for 15 seconds. 4) Drive the vehicle, repeat 2 to 3, and finish the pre-defined test route for at least one time.
2. Formula
VoLTE voice quality= total of all VoLTE MOS/quantity of completed VoLTE calls
VoLTE call setup time
1. Test Procedures 1) After the test set up ready. 2) Use the script to make a VoLTE call and then hold the call connection for 30 seconds. 3) Use the script to hang up the terminal and wait for 15 seconds. 4) Drive the vehicle, repeat 2 to 3, and finish the pre-defined test route for at least one time.
2. Formula VoLTE call setup time = total delay of all VoLTE calls/ total of all VoLTE completed calls
2.2
Golden Cluster KPIs
More than 95% sites on air without alarm in the cluster.
RRC Connection Request Success Rate 1. Test Procedures 1) After the test set up ready. 2) Start download 1 file which is 1G through internal FTP server for 60 seconds. 3) Detach terminal for 10 seconds. 4) Drive the vehicle and repeat step 2 and 3, till finish the test route for at least one time.
2. Signaling Flow chart
3. KPI Formula RRC Connection Request Success Rate= Total number of RRC Connection Request Success / Total number of RRC Connection Request *100%
ERAB drop rate (or drop call rate) 1. Test Procedures 1) After the test set up ready. 2) Start download 1 file which is 1G through internal FTP server for 60 seconds. 3) Detach terminal for 10 seconds. 4) Drive the vehicle and repeat step 2 and 3, till finish the test route for at least one time.
2. Signaling Flow chart
3. KPI Formula Main formula: ERAB drop rate (or drop call rate) = Number of drop calls * 100%/Total Number of calls For measurement, reflect to below formula: ERAB
drop
rate
(or
drop
call
rate)
=
Total
number
of
(RRCConnectionReestablishmentRequest-RRConnectionReestablishmentcomplete) *100% / Total number of Activate default EPS bearer context accept
Handover success rate 1. Test Procedures 1) After the test set up ready. 2) Start download 1 file which is 1G through internal FTP server for 60 seconds. 3) Detach terminal for 10 seconds. 4) Drive the vehicle and repeat step 2 and 3, till finish the test route for at least one time. 2. Signaling Flow chart
3.Formula Handover Success Rate = total of successful intra frequency handover operations / quantity of intra frequency handover requests * 100%
