Tuesday, October 13, 2015

Accessing files via HTTP Server

I wanted to find out better way to monitor process server logs (JVM logs) from windows environment; the Document Monitor plugin for Notepad++ utility sounded like good option, though it works the delay between refresh is unacceptable and sometimes updates are not happening until you re-open the same file.

When I'm thinking about possible alternatives, IHS came in my mind; I've seen and set up IHS to expose UNIX directories accessible via web browser. So, I just tried to replicate the same at Windows environment and it worked like a charm

Assumption: IBM HTTP Server is installed and configured for SSL

What all required?

Obviously you need to know the list of log directories/location need to be made accessible via web browser

Here I wanted to expose to Deployment Manager Logs, Node logs which includes logs for process server deployment environment

Dmgr log location	C:\IBM\WebSphere\AppServer\profiles\DmgrProfile\logs
Node log location	C:\IBM\WebSphere\AppServer\profiles\Node1Profile\logs

Things to do at windows

Create a new folder 'weblogs'
Create new empty file with name '.htaccess' and save it under weblogs folder created in previous step

Note: little bit trick involved in creating file with dot '.' as prefix;

When you're about to save empty file in notepad make sure that you select save as type as 'All Files (*.*) as shown below

Edit the '.htaccess' file and include the following lines

AuthName "Dmgr and Node logs"

AuthType Basic

AuthUserFile " C:\IBM\HTTPServer\conf\httpusers.htpasswd"

Require valid-user

What we've included above is the authentication information required to access the logs in a secure manner

httpusers.htpasswd – is the file which hold user account and password to access the logs

Create symbolic link to log directories of dmgr and node profile and save it under web logs

mklink is the command used to create Symbolic links in windows

Command syntax

mklink /D <link location> <target location>

mklink /D C:\IBM\WebSphere\weblogs\dmgrlogs C:\IBM\WebSphere\AppServer\profiles\DmgrProfile\logs

mklink /D C:\IBM\WebSphere\weblogs\nodelogs C:\IBM\WebSphere\AppServer\profiles\Node1Profile\logs

Navigate to bin directory of IHS installation and create user repository as shown below

htpasswd -c C:/IBM/HTTPServer/conf/httpusers.htpasswd psadmin

Here

httpusers.htpasswd is file to hold username and password

psadmin is the user account authorized to access the logs

-c denotes new password file created for the first time

Note: Make sure you don't specify option when adding subsequent users

Edit the httpd.conf file and add the following entries

Make sure the following module is uncommented out at 'httpd.conf' file

'LoadModule alias_module modules/mod_alias.so'

Alias /weblogs "C:\IBM\WebSphere\weblogs"

</IfModule>

Options Indexes FollowSymLinks Includes ExecCGI

AllowOverride AuthConfig

Order allow,deny

Allow from all

</Directory>

Here

/weblogs – is the alias will mention at the URL (https://youservername:securedportno/weblogs)

Restart the HTTP Server

Access the JVM logs via browser

Enter the below URL at web browser

https://processservrlocal.com:4443/weblogs

You will be prompted to enter user account and password

Enter user name/password got created at step 5

Upon successful authentication dmgr logs and node logs visisble in the browser

Following dmgr logs further you can see the JVM logs

When you want to add any other directory, just create symbolic link at weblogs directory

Tuesday, October 6, 2015

Troubleshooting tracking data transfer issue from Process Server Database to Performance Data Warehouse Database

Process Applications deployed on process server periodically sent (every 30 seconds) tracking data to performance DB

Stated below is the list of SQL Statements that you can make use to troubleshoot issues or to enable custom monitoring

Tracking data is sent from Process Server DB to PDW DB

Things can be verified at Process Server DB

1. Are there any records waiting to get transferred from BPMDB to PDWDB?

Get a row count of LSW_PERF_DATA_TRANSFER table

SELECT count(*) FROM LSW_PERF_DATA_TRANSFER

Row count shows the no of records waiting to get transferred to PDW DB

if < 500 rows, then data is most likely transferring, if not find why data is not moving

Also remember data transfer interval with respect to count

As per the design -- the data transfer happens every 30 seconds if the number of records to be transferred is less than 500. If the records are greater than 500 -- then the data transfer continuously happens until all the records are transferred

Things can be verified at Performance Data Warehouse Server DB

1. Are there any data transfer errors at PDWDB?

Get a row count of LSW_DATA_TRANSFER_ERRORS table

SELECT count(*) FROM LSW_DATA_TRANSFER_ERRORS

Row count shows the no of transfers failed due to error

Most common errors can be seen at LSW_DATA_TRANSFER_ERRORS table

(PFS-0065) The tracking group with external ID <BPD_ID> is not defined.

When tracking definitions were sent for process application which has auto tracking disabled, PFS-0065 is written to data transfer error table.
Remember auto tracking enablement with respect to BPM version

BPM versions v7.5, v8.0, v8.5 - auto tracking enabled by default

BPM v8.5.5 onwards auto tracking was disabled by default

2. Whether sending tracking definitions worked or not?

Run the SQL statement at PDWDB to identify whether tracking definitions sent successfully or not

SELECT TIME_RECEIVED, TIME_LOADED FROM LSW_LOAD_TRACE where LOAD_TYPE =

1 AND STATUS = 6

Output will show the time stamp tracking definitions were received at PDWDB

Friday, October 2, 2015

BPM Tasks and Process Instances Clean up – purging data from Process Server DB

Removing process instances from the Process Server database

Why is deleting old data necessary?

When an instance completes and all of its associated tasks are closed, future work is not possible with this instance. You cannot re-start it and assign it to someone or edit old work. When a user logs into the portal, various tables are queried to gather data on the active tasks for that user. This operation involves full table scans. Even if only 35% of the data is relevant, it is going to take a while to pull the tasks needed for that user. Thus, if the other 65% is deleted, there is less data to scan.

The cleanup functions provided in the Admin Console remove task data only, not all of the BPD instance data. TheBPMProcessInstancesCleanup command deletes both the instance and task data that is associated with the BPD. Therefore, it is a much more thorough way to clean out BPD instances. You might want your system administrator to set up a scheduled job that invokes theBPMProcessInstancesCleanup command periodically with the correct filter settings for your environment.

You can use the BPMProcessInstancesCleanup command to safely clean up data that is created by runaway processes. The command ensures that even instances with a lot of associated data are deleted in a consistent fashion.

SQL query to identify completed process instances

The following query shows the distribution of all tasks and their status.

Run this SQL query at process server DB

select code.NAME, COUNT(t.STATUS)
from lsw_task t right join LSW_TASK_STATUS_CODES code on code.STATUS_VALUE = t.STATUS
group by code.NAME
order by code.NAME

The following query shows your distribution of tasks by BPD execution status:

select inst.EXECUTION_STATUS, count(*) as total

from

LSW_BPD_INSTANCE inst,

LSW_TASK task

where task.BPD_INSTANCE_ID = inst.BPD_INSTANCE_ID

group by inst.EXECUTION_STATUS

The following query shows you the same data, but just for BPD instances:

select inst.EXECUTION_STATUS, count(*) as total

from

LSW_BPD_INSTANCE inst

group by inst.EXECUTION_STATUS;

select code.NAME, COUNT(bpd.EXECUTION_STATUS)

from LSW_BPD_INSTANCE bpd right join lsw_bpd_status_codes code on code.STATUS_ID = bpd.execution_status

group by code.NAME

order by code.NAME;

BPMProcessInstancesCleanup command

Use this command to delete business process definition (BPD) instance data for a process application snapshot on IBM® Process Server.

Purpose

Using this command deletes the BPD instance and its associated tasks for the instances that are specified by the command parameters. It also logs data to a standard systemout.log file to track which process applications were selected for deleting instance data.

To run this command, you must be in the tw_admins group. Also, you can run this command only in the connected mode. In a network deployment environment, you must run this command on the node containing the application cluster member that handles Process Server applications. Do not run this command from the deployment manager profile.

Restriction: The following restrictions apply:

You cannot use this command to delete BPEL process instances.
You can run the command from any cluster member in a network deployment environment. However, you must first establish the wsadmin session to the SOAP port of the cluster member from where you are running the command.

Parameters

-containerAcronym containerAcronym

A required parameter that identifies the acronym that specifies the process application to perform process instance cleanup against. For example, the BillingDispute process application might have an acronym of BILLDISP.

-containerSnapshotAcronym containerSnapshotAcronym

A required parameter that identifies the snapshot acronym for the process application. The snapshot acronym can be different from the snapshot name. To find the snapshot acronym, run the BPMShowProcessApplication command. Refer to the topic in the related link.

-instanceStatus instanceStatus

A required parameter that identifies the status of process instances to clean up.

COMPLETED - Removes all completed process instances
FAILED - Removes all failed process instances
CANCELED - Removes all terminated process instances
ALL - Removes all completed, failed, and canceled process instances

Tip: You cannot delete an instance that is currently running.

-instanceID instanceID

An optional parameter of type String[] that identifies a list of instance IDs that are being removed. If there is more than one instance ID, each instance ID must be delimited by a space for Jython format commands or a semicolon for JACL format commands. You can also use this parameter withendedAfterLocal or endedBeforeLocal. See the examples in this topic.

-endedAfterLocal endedAfterLocal

An optional parameter of type String that identifies the local time on the server. The string must have the following format:yyyy-MM-ddThh:mm:ss (year, month, day, T, hours, minutes, seconds). Only instances that completed, failed, or canceled after the specified time period by this parameter are processed. If you use this parameter with the endedBeforeLocal parameter, only instances that completed, failed, or canceled during the specified time period by this parameter are processed.

-endedBeforeLocal endedBeforeLocal

An optional parameter of type String that identifies the local time on the server. The string must have the following format: yyyy-MM-ddThh:mm:ss (year, month, day, T, hours, minutes, seconds). Only instances that completed, failed, or canceled after the specified time period are processed. If you use this parameter with the endedAfterLocal parameter, only instances that completed, failed, or canceled during the specified time period by this parameter are processed.

-outputFile outputFile

An optional parameter that provides a qualified file path to write the process instance cleanup log entries. The default is SystemOut.log.

Sample execution

Run sql query to determine the no of completed BPD instances

Here 6025 instance is in completed state

Connect to soap port of application cluster member

C:\IBM\WebSphere\AppServer\bin>wsadmin -conntype SOAP -port 8882 -host bpmprocessserver.local.com -user psadmin -password psadmin

WASX7209I: Connected to process "PSDE1.SingleCluster.processserverlocalNode03.0" on node processserverlocalNode03.0 using SOAP connector; The type of process is: Man

agedProcess

WASX7029I: For help, enter: "$Help help"

wsadmin>

Run BPMProcessInstancesCleanup

$AdminTask BPMProcessInstancesCleanup {-containerAcronym HSAV1 -containerSnapshotAcronym HSAV856 -instanceStatus ALL -endedBeforeLocal 2015-10-02 -outputFile C:\IBM\WebSphere\AppServer\HSAV1-processinstancecleanup_02Oct15.txt}

Here

-containerAcronym

found by “ $AdminTask BPMListProcessApplications”

Note:each process apps will have an Acronym

-containerSnapshotAcronym

found by “$AdminTask BPMShowProcessApplication {-containerAcronym “value1”}”

value1 – containerAcronym of process application

Note: each process apps have no of snapshots associated

Check the output file

[2015-10-02 16:04:31.197] About to start the BPMProcessInstancesCleanup with parameters: -containerAcronym Hiring Sample Advanced -containerSnapshotAcronym HSAV856 -instanceStatus ALL -endedBeforeLocal 2015-10-02 00:00:00.0 -maximumduration 0 -transactionSlice 1 -outputFile C:\IBM\WebSphere\AppServer\HSAV1-processinstancecleanup_02Oct15.txtSuccessfully ran the BPMProcessInstancesCleanup command with parameters: -containerAcronym Hiring Sample Advanced -containerSnapshotAcronym HSAV856 -instanceStatus ALL -endedBeforeLocal 2015-10-02 00:00:00.0 -maximumduration 0 -transactionSlice 1 -outputFile C:\IBM\WebSphere\AppServer\HSAV1-processinstancecleanup_02Oct15.txt

Number of qualifying instances before deletion: 30

Number of qualifying instances after deletion: 0