For internal use 1 / 25
Performance Management NetAct Administration course (NetAct 16.2)
company.nokia.com
© Nokia 2014
For internal use 2 / 25
This material, including inc luding documentation and any related computer programs, is protected by copyright controlled by Nokia. All rights are reserved. Copying, including reproducing, storing, adapting or translating, any or all of this material requires the prior written consent of Nokia. This material also contains confidential information, which may not be disclosed to others without the prior written consent of Nokia.
company.nokia.com
© Nokia 2014
For internal use 3 / 25
Contents 1
Overview and Architecture ............................................................................................................................... 5
1.1
Introduction ...................................................................................................................................................... 5
1.2
Performance Manager functional architecture ....................................................................................... 6
1.3
NetAct 16.2 PM functions.............................................................................................................................. 7
1.4
Performance Manager UI .............................................................................................................................. 9
2
PM Platform Implementation .......................................................................................................................... 11
2.1
VM node configuration .................................................................................................................................. 12
2.2
PM processes ................................................................................................................................................... 13
3
PM Functionality .................................................................................................................................................. 14
3.1
ETLOAD.............................................................................................................................................................. 14
3.1.1
Data collection .............................................................................................................................................. 14
3.1.2
Data loading .................................................................................................................................................. 15
3.1.3
Data aggregation ......................................................................................................................................... 16
3.2
Database cleanup............................................................................................................................................ 17
3.3
Partitioning........................................................................................................................................................ 18
4
PM Administration .............................................................................................................................................. 19
4.1
Administration of Measurements ............................................................................................................... 19
4.2
Configuring the aggregation process ........................................................................................................ 19
4.2.1
Enabling parallelization ............................................................................................................................... 19
4.2.2
Selective summarization ............................................................................................................................ 20
4.2.3
Trace configuration ..................................................................................................................................... 20
4.2.4
Skipping hourly aggregations ................................................................................................................... 20
4.2.5
Manual update of Oracle statistics on Topology Database ............................................................. 20
4.3
Data reliability................................................................................................................................................... 20
4.4
Configuring PM Objects replication period ....................................... ....................................................... 21
4.5
Configuring data extraction and loading ........................................... ....................................................... 21
4.5.1
Working with OMeS backup files .............................................................................................................. 21
4.5.2
Configuring the etldanmx processes running in parallel .............................................................. 22
4.5.3
Configuring the etlpslmx processes running in parallel .............................................................. 22
company.nokia.com
© Nokia 2014
For internal use 4 / 25
4.5.4
Discarding duplicate measurements ...................................................................................................... 22
4.5.5
Discarding the delayed data ..................................................................................................................... 22
4.5.6
Number of files etlcolmx can process before alarm .................................................................... 22
4.5.7
Discarding delayed notifications at hourly level ............................................................ ...................... 22
4.5.8
Disabling OLAP notification calls .............................................................................................................. 22
4.5.9
Configuring etldanmx process to compress notifications ........................................................... 22
4.5.10
Configuring support for repN scalability in cosprc....................................................................... 23
4.5.11
Configuring capacity logging .................................................................................................................. 23
5
Basic Monitoring and Troubleshooting ......................................................................................................... 24
5.1
Log files and tracing ....................................................................................................................................... 24
5.1.1
PM platform logs.......................................................................................................................................... 24
5.1.2
ETLOAD process tracing ............................................................................................................................ 24
5.1.3
Other log and trace files ............................................................................................................................ 24
5.2
Troubleshooting documentation ................................................................................................................ 25
company.nokia.com
© Nokia 2014
For internal use 5 / 25
1 Overview and Architecture 1.1
Introduction
The aim of any performance management activity is to collect data to support the following activities:
verifying the physical and logical configuration of the network monitoring continuously how the network functions localizing potential problems as early as possible monitoring subscriber behavior providing optimum services to mobile subscribers
The performance management functionalities in NetAct 16.2 are made up of an extended family of applications for processing, analyzing, and visualizing performance data that is coming from different sources. Raw data becomes meaningful information that is visualized in graphical and textual reports. Performance Management in NetAct 16.2 gives a view of the network and service performance and makes it possible to analyze network data, create reports based on the data and distribute the information within the operator’s organization.
Umbrella Network Management
Domain Management
Umbrella FM
Umbrella CM
FM
CM
…
PM
NetAct OSS5.x
Element Management
Umbrella system i
NPM
MV EMS 1
PM
FM
…
MV EMS i
OMS1
CM NetAct 8
OMSi
OMS2
Network Elements Radio access
7
1 9 / 0 6 / 2 0 1 4 © Nokia 2014
Broadband access
IP Transport
Converged Core
1.0 MBB- CS- OSS- CoDe
For internal use
In NetAct 16.2, the Performance Management (PM) platform is called NetAct Performance Manager (a.k.a OnePM). Performance Manager is multi vendor-capable and collects data from the entire network that consists of network elements from Nokia Siemens Networks and from other vendors. It processes and stores the data, from days to years, depending on your requirements, and also includes so several ready-made report packages.
company.nokia.com
© Nokia 2014
For internal use 6 / 25
1.2
Performance Manager functional architecture
The functional architecture of Performance Manager can be divided into 3 layers:
integration layer
process layer
reporting layer
The figure below is generic for both NSN PM and NetAct PM. Some of the functions may not be applicable for NetAct PM.
PM Portal PM portal
NBIs Advanced content
Pre-defined content
Scheduling
Aggregations
SBIs
Radio access
1 9 / 0 6 / 2 0 1 4 © Nokia 2014
Dashboards
PM alarm s
Administrat ion
Loading
GIS reporting
Data/Metadata
Enrichment
SBIs
PM/CM/Topology (file, SNMP, SQL…)
EMS
8
Ad-hoc reports
Transformatio n
Collection
NBIs
EMS
EMS
Broadband access
IP Tr an sp or t
Co nve rg ed Co re
1.0 MBB- CS- OSS- CoDe
For internal use
The
is responsible for all ETL (extraction, transformation and loading) related
activities:
Collection uses mediation components that can connect to the integrated EMS (element management system), e.g., via ftp/sftp/SNMP, and collect PM/CM/Topology data. Push operations can also be used.
Transformation adapts the vendor specific formats and protocols to PM internal xml based format (OMeS).
Loading takes the PM data in OMeS format and stores it in the data warehouse.
Enrichment takes care of managing & storing other types of data that can be used together with PM data (CM/FM statistics, GIS coordinates, etc.) on the reporting layer.
The
takes care of data preparation and summarization:
Scheduling enables the administrator to increase system efficiency, by scheduling most heavy reports during offline hours.
company.nokia.com
© Nokia 2014
For internal use 7 / 25
Raw data is aggregated in object/time dimension and stored according to customer preferences, allowing long-term and fast reporting. BH (Busy Hour) calculations from several algorithms are also provided.
PM alarms can be generated based on KPI thresholds and/or KPI profiling deviations, using Thresholder & Profiler tool.
Administration tools includes toolkits that allows to e.g., monitor system health and act upon abnormal behaviour, change system configuration, troubleshoot data availability issues or even extend default data models.
Data warehouse is based on Oracle database (w/ RAC option on project basis) and is used to store raw/aggregated PM data, pre-calculated KPI Values and other relevant information (e.g., topology, CM statistics, etc.). Metadata is stored in DW or file system and transversely serves all system tools, from mediation to reporting layer.
The
includes:
pre-defined content packages incorporating NSN telecommunications experience and know-how, in the form of useful KPIs and reports for major technology areas
advanced Content packages providing valuable vendor and technology agnostic KPIs and dashboards
ad-hoc reports, which can be created out of regular network counters, pre-defined or custom made KPIs
dashboard framework, which allows users to tailor their own reports, with focus on presentation and layout capabilities
GIS reporting introduces specific map views, where meaningful KP Is and objects can be represented in geographical way
1.3
NetAct 16.2 PM functions
The figure below shows the various PM functions in NetAct 16.2:
company.nokia.com
© Nokia 2014
For internal use 8 / 25
OMeS File
Administration of aggregations
Reporter GUI
NB PM mediation component
RTPM GUI
Administration of measurements
Reporting request
Aggregation request
Reporting engine AoM request
PM Query
Query engine Measurement aggregation Info about new measurement
DB Query
PM DB
Aggregated measurements Measurement to b e stored
ET Load Profiler and Thresholder
OMeSfile
Mediation (*) SB PM Mediation component Raw PM
9
1 9 / 0 6 / 2 0 1 4 © Nokia 2014
AoM Backend
AoM request to NE
1.0 MBB- CS- OSS- CoDe
For internal use
– This application manages the collection of measurements from network elements to NetAct. It is used to view, create, modify and delete measurement plans and templates. – There are several southbound mediation components that are responsible for retrieving the raw measurement data from the various network elements integrated to NetAct. – picks up the OMeS files from the PM Southbound Mediation Interface and loads these measurements into the PM database. – After the measurements are loaded, they can be processed further through aggregation. – Once the measurements are in the DB, they are available for the reporting tools. Using the Reporter GUI, reports can be created with a granularity of 15 minutes. – is responsible for generating the actual reports. – is responsible for running the SQL queries and returning the results to the Reporting engine – Real-Time Performance Monitoring is a new tool introduced in NetAct 16.2. It was already available in NetAct 7 but the PM platform has changed and it needs to be adapted to NetAct 16.2. It provides a view of performance information in near real time with a granularity of minutes.
company.nokia.com
© Nokia 2014
For internal use 9 / 25
1.4
Performance Manager UI
The Performance Manager web-based user interface includes many different applications and tools.
10
1 9 / 0 6 / 2 0 1 4 © Nokia 2 01 4
1 .0 MBB- CS- OSS- CoDe
For internal use
– In the
area all the reports saved by any user are available to be
reused. You are able to see your own report navigation tree and also the reports that have been shared by other users. The tree can be customized by every user. – This view displays the default content delivered by Nokia Solutions and Networks. Only administrators can customize this content, create and publish shared reports in the tree for all users. The customization is performed via
. Also, only the administrator can
create a view and publish shared reports in the tree for all users. – The
tool is used to create new reports in a fast and easy way, based
on technologies supported by Nokia Solutions and Networks. – The
tool is used to create new KPIs in a fast and easy way, based on
technologies supported by Nokia Solutions and Networks. – The
area can only be accessed by the system
administrators. In this area administrators are able to publish the shared reports making them available to all users in the Reporting Suite tree, which structure can also be cu stomized. – The
area can only be accessed by the administrators. In this area
administrators are able to publish the private KPIs, making them available to all users.
company.nokia.com
© Nokia 2014
For internal use 10 / 25
– Reporter KPI Converter is an extension to Report Builder's formula editor. It is meant for sites that have the Reporting Suite software installed. It imports typical Reporting Suite KPI definitions into the Report Builder PI tree as formulas. – The
area can only be accessed by the administrator. In order to allow the
control of system resources, the administrator can run, pause, delete and modify existing jobs. – Reporter Admin Toolkit allows you to check the system status, and monitor and configure the Netact PM pipe. Note: Admin Toolkit is not fully implemented in NetAct 16.2 LTE. In the current build, there is only Adaptation Configuration and Storing Period Configuration. Eventually (hopefully in NetAct 16.2), there will also be Health Check, Log File Viewer, Trace File Viewer, WebSphere Status, Data Reliability, Partition Manager Log Viewer and Mediation Configuration. – Thresholder and Profiler are NetAct applications that are integrated with NetAct Performance Manager: they allow you to set and manage thresholds or profiles, respectively, for the performance indicators (PIs) calculated in Performance Manager. – The Thresholder Admin Toolkit is only available to users who have the Reporter_Admin permission. It allows you to configure Thresholder and manage the scheduled Thresholder jobs.
company.nokia.com
© Nokia 2014
For internal use 11 / 25
2 PM Platform Implementation The figure below shows the general implementation of NetAct Performance Manager in NetAct 16.2:
NetAct start page integration (SSO/PEM/UM)
AdminToolkit
Reporting tools
Aggregations
Reporting Engine
OMeS, 3GPP
6
AoM
Northbound PM mediation
OIB
…
5
Function(i)
…
PM access/meta WS
3
Application (i)
Object Register
Function (i+1)
Application (i+1)
NMI (OMeS 2.1/2.3)
Loading
NASDA
NetAct PM / OnePM SMI
2
OMeS 2.1/2.3
1
PM Mediation (i)
PM Mediation (i+1)
NetAct DB
PM access SQL
PM
CM
FM
…
4
Other Mediations (i)
Other Mediations (i+1) NetAct 8 LTE
12
1 9 / 0 6 / 2 0 1 4 © Nokia 2014
1.0 MBB- CS- OSS- CoDe
For internal use
1. PM mediations send OMeS files to the PM South Mediation Interface. 2. PM loading component parses OMeS 2.1 or 2.3 files, inserts into PM database and makes the files available again in PM North Mediation Interface (for use by NB mediation components). 3. PM MO creation is synchronized with NASDA (NetAct System Data Access) Object Registry (same GIDs as NetAct 16.2 objects). 4. PM database exposes PM access SQL-based interface for fetching raw/aggregated measurement results. 5. Reporting Engine makes available PM access and PM meta access WS-based APIs, allowing Report/KPI CRUD (create, read, update and delete) and execution. 6. PM reporting tools are published in the NetAct 16.2 start page and SSO is possible between the applications. NetAct PM User & Permission Management is replaced by NetAct 16.2 UM/PEM.
company.nokia.com
© Nokia 2014
For internal use 12 / 25
2.1
VM node configuration
NetAct Performance Manager is fully virtualized along with the rest of NetAct 16.2. The exact configuration depends on the NetAct 16.2 configurations: small, mainstream and large.
=6vCPU
=20vCPU
=14GB vRAM
=48GB vRAM
=200GB VMDK
=400GB VMDK
Small 14
1 9 / 0 6 / 2 0 1 4 © Nokia 2014
Mainstream 1.0 MBB- CS- OSS- CoDe
For internal use
1
WAS 6.1 cluster hosting reporting applications
2
VM hosting OSS5.x DS services (ldap, dns, etc.)
3
Message
4
VM hosting OSS5.x Connectivity server services (apache, topology, etc.)
5
VMs hosting OnePM loading and aggregation services
WebSphere AppSrv
VM 4
12
VM 60
4
12
NFS
VM 60
4
60
4
12
4
3
DB_CRONS
60
5
CMESSAGE
DIR
1
OSSCORE
DNS
2
12
VM
MESSAGE
HTTP
REP
REP
REP
BASE
BASE
BASE
BASE
=36vCPU =96GB vRAM =660GB VMDK
VM 2
15
4
VM 70
4
1 9 / 0 6 / 2 0 1 4 © Nokia 2014
4
VM 70
2
4
VM 70
4
12
VM 70
4
12
VM 70
4
12
70
1.0 MBB- CS- OSS- CoDe
For internal use
1. WAS 6.1 cluster hosting reporting applications 2. VM hosting OSS5.x DS services 3. Message company.nokia.com
© Nokia 2014
For internal use 13 / 25
4. VM hosting OSS5.x Connectivity Server services 5. VMs hosting OnePM loading and aggregation services 2.2
PM processes
The table below includes a short description of the PM platform processes along the their werlog code (werlogXXXX.log): admlogmx
Log parser process for werlog and
A6B8
admoralogmx
Oracle log parser for alertoss.log
A6B8
admlogcleanmx
Log cleanup process from DB
A6B8
admsupmx
Admin Toolkit supervisor
A6C6
cosprcmx.pl
Performs aggregations and sends to NMI
A308
e3sofcmx
System OMeS file converter
A448
e3sotomx.pl
Special OMeS-to-OMeS file converter
A485
etlbkkcleanmx.pl
Registration table cleanup
A121
etlcolmx
OMeS data file collector
A51B
etldanmx
PM data arrival notification client
A3FB
etlexpmx
OMeS file export filtering process
8890
etlpslmx
OMeS files parser and loader
A51A
pmicvtmx.pl
PM adaptation converter
A2E2
pmiexpmx.pl
PM adaptation data exporter
A2E3
pmifiamx
PM adaptation FTP information access
A3B9
pmiimpmx.pl
PM adaptation data importer
A2E4
pmisysmx
PM adaptation controller
A2E7
pmpmgrmx
PM DB partitioning manager
A66A
pmpcnfmx
PM DB partitioning management configurator
A66B
spmcaplogparsermx SelPM capacity log parser spmclnmx SelfPM log cleanup
company.nokia.com
A6C2 A6C7
© Nokia 2014
For internal use 14 / 25
3 PM Functionality 3.1
ETLOAD
Performance Manager includes a subsystem, PM Data Extraction, Transaction, and Loading (ETLOAD), that inserts the raw data that was uploaded from the NEs into the Performance Manager database. ETLOAD takes the OMeS files as input, fetches the data contained in the OMeS files, and loads the data into adaptation-specific tables in the database.
1) Measurement data is collected from the NEs through the respective Southbound Mediations. 2) PM SMI collects OMeS files from the Southbound Mediations. 3) PM ETLOAD inserts the measurements to NetAct database. 4) Raw measurement files are exported to external systems in OMeS format via PM NMI. 5) Measurements are aggregated by the Generic Process for PM, cosprcmx.pl. 6) Performance Manager displays/generates performance management data.
20
1 9 / 0 6 / 2 0 1 4 © Nokia 2 01 4
1 .0 MBB- CS- OSS- CoDe
For internal use
1. Measurement data is collected from the NEs through the respective Southbound Mediations. 2. PM SMI collects OMeS files from the Southbound Mediations. 3. PM ETLOAD inserts the measurements to NetAct database. 4. Raw measurement files are exported to external systems in OMeS format via PM NMI. 5. Measurements are aggregated by the Generic Process for PM, cosprcmx.pl. 6. Performance Manager displays/generates performance management data. 3.1.1
Data collection
In each REP VM, there are 4 program groups responsible for the ETLOAD process. There are 4 instances of the etlcolmx process on each REP VM, 1 for each program group. Each etlcolmx process is responsible for collecting OMeS files.
company.nokia.com
© Nokia 2014
For internal use 15 / 25
$VARROOT/rep/etload/pg_repetl/work/ download load_1 etlcolmx_rep1
load_2 load_3 load_4
etlcolmx_rep1_1
$VARROOT/rep/etload/pg_repetl_1/work/ etlcolmx_rep1_2
$VARROOT/rep/etload/pg_repetl_2/work/ etlcolmx_rep1_3
$VARROOT/rep/etload/pg_repetl_3/work/
$VARROOT/osscore/iscirp/pmupld_osscore/ 22
1 9 / 0 6 / 2 0 1 4 © Nokia 2 01 4
1 .0 MBB- CS- OSS- CoDe
For internal use
The 4 etlcolmx processes on each REP VM take turns (in a loop or round robin basis) collecting OMeS files from the pmupld_osscore and transferring them to its program group's work folder. The OMeS files are first moved to the download folder and then distributed among the load folders where they are then picked up by etlpslmx processes dedicated for each load folder. The etlcolmx processes across all REP VMs poll the same pmupld_osscore folder and they collect the OMeS files on a first-come first-serve basis. 3.1.2
Data loading
The process responsible for picking up OMeS files from the load directories and inserting the RAW data into the PM DB is etlpslmx. By default, there are four etlpslmx processes for each program group. Since there are 4 program groups on each REP VM, it means that there are a total of 16 etlpslmx processes by default.
company.nokia.com
© Nokia 2014
For internal use 16 / 25
writ e filt ered OMeSfiles to out put directory
etldanmx_rep1 read not ification files
notify
etlexpmx_rep1
write notif ication files
read OMeSfi les for export
export
insert t o ETL_P_RAW_DATA_BOOK
writ e OMeS files (wit h DN)
etlpslmx_rep1_1
read OMeSfi les
load_1
insert t o RAWt ables
PM DB
writ e OMeS files (if configured)
backup
$VARROOT/rep/etload/pg_repetl/work/ 23
1 9 / 0 6 / 2 0 1 4 © Nokia 2 01 4
1 .0 MBB- CS- OSS- CoDe
For internal use
The number of etlpslmx processes for each program group can be configured from 1 up to 9 using the etlpslReconfigProcessNumber.pl script. Each etlpslmx instance has its own dedicated load folder located in its program group’s work folder. Each etlpslmx process will read the OMeS files from its own load folder and insert the raw measurements to the corresponding PM DB tables. After inserting data to the PM DB, it will write a notification file to the notify folder with information about what it has just processed.
The
etldanmx processes will then read the notification files from the notify folder and insert the
information to the ETL_P_RAW_DATA_BOOK table. If the data reload feature is enabled, the processed OMeS files will be written to the backup folder. By default, the OMeS files are kept in the backup folder for 7 days. If export of measurements is enabled, the processed OMeS files will be written to the export folder where they will then be picked up by the etlexpmx processes. If there are temporary failures, the OMeS files are written to the temp_err folder. They’ll be picked up again later by the etlcolmx process and redistributed to the load folders. By default OMeS files in temp_err are kept for 14 days. If there are permanent failures, the OMeS files are written to the perm_err folder. By the default OMeS files in the perm_err folder are kept for 14 days. 3.1.3
Data aggregation
The cosprcmx.pl script takes care of aggregations. It runs as a cron job for each adaptation. It analyzes the ETL_P_RAW_DATA_BOOK to figure out which measurements to aggregate.
company.nokia.com
© Nokia 2014
For internal use 17 / 25
etldanmx_rep1
export
insert t o ETL_P_RAW_DATA_BOOK write export files analyze ETL_P_RAW_DATA_BOOK read RAW& summary tables
ft p export files t o GC/NPM
cosprcmx.pl
read met adata
PM DB insert t o summary tables
Metadata configuration files
meta_mx.cf
24
1 9 / 0 6 / 2 0 1 4 © Nokia 2 01 4
1 .0 MBB- CS- OSS- CoDe
For internal use
The cosprcmx.pl script reads the metadata configuration file for the corresponding adaptation(s). These metadata configuration files contains the formulas and other relevant information for performing the aggregations. The cosprcmx.pl script then reads the necessary data from the raw and summary tables and inserts the aggregated data to the corresponding summary tables. If export of aggregates is enabled, it writes the files to the export folder. If aggregates are to be exported to the Global Cluster the files are exported via ftp. 3.2
Database cleanup
Cleanup of measurements in the database are carried out by the following programs:
pmpmgrmx.pl – cleanup for partitioned measurement data etload_pm_meas_clean.pl – cleanup for non-partitioned raw measurement data (e.g. AXC raw measurements) cosprcmx.pl – non-partitioned summary measurement data (e.g. AXC summary measurements)
The above programs are generally activated as cron jobs for the omc user but can also be invoked via command line. Automatic cleanup of measurements can be configured in the file psrsurvivalmx.cf. Instructions can be found in the Operating Documentation under .
company.nokia.com
© Nokia 2014
For internal use 18 / 25
3.3
Partitioning
Partitioning Manager maintains the partitioned measurement tables. The expired data is cleaned up with partition dropping. The usage of partitioning in a measurement table makes a large table more manageable when only a part of the data is the target and it is delimited by the measurement time period. Partitioning is useful for large volumes of data. Also, if a measurement table contains historical data, and new data is added into the newest partition, it means that a rolling window of data is stored. Local index partitioning is applied for the indexes of the measurement tables. A local index is automatically partitioned in the same way as the table is partitioned. New internal alarms are raised and cancelled by PM DB Partitioning Manager. The benefits of partitioning are:
Clean-up of expired data is faster when a partition gets dropped instead of data being deleted. Query performance is better. The results of a typical query of PM data can be achieved by accessing a subset of partitions rather than the entire table. PM data insertion into a partitioned table is faster. Index rebuilding can be done for partitions and not the whole index.
company.nokia.com
© Nokia 2014
For internal use 19 / 25
4 PM Administration 4.1
Administration of Measurements
Administration of Measurements (AoM) is an application in NetAct that assists you to manage and administer measurements in network elements, and provides the function to upload the measurements configuration of network elements to NetAct manually. With Administration of Measurements, you can view, create, edit, delete and get the detailed information of measurement templates and plans. And you can export the template data to local disk, activate and deactivate the plans.
To launch Administration of Measurements, login to the NetAct Start Page and under the Reporting category, click the Administration of Measurements icon. To perform all the available functions, the user must have the following permission operations from the AoM permission object:
4.2
Launch AoM Plan Management AoM Template Management
Configuring the aggregation process
Instructions for configuring the aggregation process are found in the Operating Documentation under . 4.2.1
Enabling parallelization
company.nokia.com
© Nokia 2014
For internal use 20 / 25
This is configured in each adaptation's respective sys_mx.cf file. Use the PARALLEL_PROCESS_NR parameter to determine how many parallel processes are required.
4.2.2
Selective summarization
The MaxObjSelSumm parameter limits the number of measuring object IDs used in the query for a selective summarization. This check is required if there is data from many measuring objects to be summarized selectively. The parameter is manually added in the file: /etc/opt/nokia/oss/reporter/conf/reporter.cf
The _SELECTIVE_SUM_PCT parameter defines the selective summarization percentage. The parameter is added manually to the adaptation system configuration file. 4.2.3
Trace configuration
The optional TRACE_LEVEL parameter defines the level of trace to be generated. The default level is 5.
To modify trace configuration, copy the relevant sys_mx.cf from
$ETCROOT//conf to $ETCROOT/custom/conf.
4.2.4
Skipping hourly aggregations
The aggregation engine computes the summarization at hourly, daily, weekly, and monthly levels, computing first the hourly summary by considering all the raw data coming at 15 minutes interval which arrives from the 0 minute until 59th minute for that particular hour. If data arrives every 60 minutes, instead of the usual 15 minutes, the aggregation engine calculates the hourly summarization for the incoming data after every 60 minutes which is the same as the raw data. Hence there is the need to skip the hourly aggregation by calculating the daily summary from hourly view and skipping the calculation of the hourly summary. 4.2.5
Manual update of Oracle statistics on Topology Database
Oracle statistics for the topology database tables are automatically updated around 6AM by a cron job. These topology tables are inherently referred during PM Aggregate/Report computations. However sudden increase of PM data load can lead to aggregates/reports not being computed and even high DB CPU load. This is better noticed if the time until the next run of the cron job takes some hours. In order to avoid this, manual update of database statistics on the topology tables should be executed. 4.3
Data reliability
The data reliability feature provides important statistical data that allows you to identify if there were gaps in the data flow process. These gaps must be taken into consideration when performing corrective actions, since reports are often used in the decision making process. Knowing the data reliability for a specific report is very important. company.nokia.com
© Nokia 2014
For internal use 21 / 25
Each measurement reliability, for each day, is taken into consideration when calculating the report data reliability. Data reliability is not available for transient levels. You must verify the data reliability for the physical level closer to that transient level. For reports using the week aggregation level, the data reliability evaluates the data for the day level, assuming the same data volume for the upper aggregates. Instructions on deactivating and reactivating Data Reliability are found in the
Operating
Documentation under . 4.4
Configuring PM Objects replication period
PM Objects replication is a process to collect additional attributes from CM tables like presentation name for Managed Objects/Network Elements. PM platform creates topology out of the OMeS file in to the PM topology tables, but not all object attributes can be derived from there, for example, object name, operation status, etc. This is about fetching this information from CM platform and trigger the update on PM topology. By default PM replication is configured to run every 10 minutes. However, that value can be configured in the Topology Replication component. Instructions on configuring the PM Objects replication period are found in the Operating Documentation under . 4.5
Configuring data extraction and loading
Detailed procedures of all the following are found in the Operating Documentation under
4.5.1
Working with OMeS backup files
Each measurement file that is successfully loaded into the database can be compressed and stored in backup directories.
This feature is enabled in the etlpslmx.cf file (copied to
$ETCROOT/custom/conf) by setting EnableDataBackup to true.
We can define how long the measurements are kept in the backup folders in the etloadmx.cf file (copied
to
$ETCROOT/custom/conf)
by
changing
the
value
of
BackupDirectoryCleanupPeriodInDays.
If you wish to reload the measurements from the backup folder, simply copy the measurement files to the download folder.
company.nokia.com
© Nokia 2014
For internal use 22 / 25
4.5.2
Configuring the etldanmx processes running in parallel
It's possible to configure the number of etldanmx processes for each program group, in order to speed up the notification process. This is done using the etldanReconfigProcessNumber.pl script (run as omc user). 4.5.3
Configuring the etlpslmx processes running in parallel
It's possible to configure the number of etlpslmx processes for each program group, in order to speed up the loading of raw measurements to the database. This is done using the etlpslReconfigProcessNumber.pl script (run as omc user). 4.5.4
Discarding duplicate measurements
The etlpsl process inserts data into the database using a volume insert process for every measurement. When the process fails, it follows the r ow-by-row approach (by default) or by the temp table approach (depending on the configuration setting made in etlpslmx.cf). It's possible to configure the etlpsl process to stop using the volume insert process and stop inserting the same measurement. 4.5.5
Discarding the delayed data
It's possible to discard the delayed data based on delayed threshold by configuring the etlpslmx.cf file.
4.5.6
Number of files etlcolmx can process before alarm
The etlcolmx process collects OMeS files for processing from different work load directories. If the number of files in those directories is higher than a specific limit, then an alarm is raised. The same happens if the number of files in error directories exceeds a defined limit. 4.5.7
Discarding delayed notifications at hourly level
The aggregation of the delayed notification has an impact on the system performance. In order to improve the system performance, the delayed notification can be discarded at the hourly level by configuring the DELAYEDDATA_THRESHOLD parameter in the adaptation configuration file. 4.5.8
Disabling OLAP notification calls
By default the OLAP notifications functionality is activated, enabling the update of the persistent reports every time that new or delayed data arrives. It's possible to deactivate this functionality by configuring the IsOLAPEnabled tag to false in the reporter.cf file.
4.5.9
Configuring etldanmx process to compress notifications
It's possible to configure the etldanmx process to compress the duplicated notifications without losing the required information. company.nokia.com
© Nokia 2014
For internal use 23 / 25
4.5.10
Configuring support for repN scalability in cosprc
It's possible to configure the repN packages in order to have scalability in cosprc process. 4.5.11
Configuring capacity logging
By default, Performance Management Capacity logging is enabled and the logs are kept for 15 days. This can be changed in the etlexpmx.cf file.
company.nokia.com
© Nokia 2014
For internal use 24 / 25
5 Basic Monitoring and Troubleshooting 5.1 5.1.1
Log files and tracing PM platform logs
Log files for the various PM processes are located in the REP VMs under $OMCLOGDIR: /var/opt/nokia/oss/local/common/log/
There is a table of werlog files in the Operating Documentation found under
5.1.2
.
ETLOAD process tracing
When etload process failures are found, traces should be activated in order to get more information about the root cause of the problem. Note, however, that the writing of trace logs may increase the load of the system considerably, especially the I/O load. Use tracing only for troubleshooting purposes and disable it when the problem has been solved. Tracing can be enabled via the respective configuration files for the etlcolmx, etlpslmx, etldanmx and etlexpmx processes. The original configuration files are found under $ETCROOT/reporter/conf. They should first be copied to $ETCROOT/custom/conf (if not done already) and edited there. Once enabled, the trace files can be found under $VARROOT/local/common/tmp/trace Detailed instructions are in the Operating Documentation under . 5.1.3
Other log and trace files
Capacity logs can be found under: $VARROOT/local/common/tmp/
WebSphere log and trace files can be found under: /var/opt/Nokia/log/
Oracle trace files can be found at: /opt/oracle/diag/rdbms/oss/oss/trace
company.nokia.com
© Nokia 2014
For internal use 25 / 25
5.2
Troubleshooting documentation
There is an official troubleshooting document include in the NetAct 16.2 Operating Documentation. PM related troubleshooting instructions can be found under
.
company.nokia.com
© Nokia 2014