Friday, 21 September 2018

Zabbix API create item using CURL

Zabbix allow to create and configure items using API. It faster than clicking using GUI.

On this example I use CURL to call api.
First you need to generate token to identifying you credentials.



Check API version

curl  -H "Content-Type: application/json-rpc" -X POST  http://zabbix/zabbix/api_jsonrpc.php -d '{"jsonrpc":"2.0","method":"apiinfo.version","id":1,"auth":null,"params":{}}'


Generate API token


curl -i -X POST -H 'Content-type:application/json' -d '{"jsonrpc":"2.0","method":"user.login","params":{ "user":"adm***","password":"****"},"auth":null,"id":0}' http://zabbix/zabbix/api_jsonrpc.php


Check host ID not using API


Login to zabbix and show configuration of you host, ID of this host will be in URL address

It is a simplest way to find your host ID.


Prepare new item using existing item 


Creating item require id of host and id of interface which is used to check item.
When you know the host ID you can create simple item from interface and next create item based on existing item.

To get configuration of item you need to specify item.key

Simple json file to call api:

{
    "jsonrpc": "2.0",
    "method": "item.get",
    "params": {
        "output": "extend",
        "hostids": "10336",
        "search": {
            "key_": "system.uptime"
        },
        "sortfield": "name"
    },
    "auth": "193ab2b48e6c428477869f6dcfa4dd26",
    "id": 1
}

10336 - is your hostid
system.uptime - is you item.key which you want to show
auth - your token genarated in step generate api token.

Save you file as js.json and call

curl -i -X POST -H 'Content-type:application/json' -d @js.json http://zabbix/zabbix/api_jsonrpc.php



Your output should look like this

{
  "jsonrpc": "2.0",
  "result": [
    {
      "itemid": "54816",
      "type": "0",
      "snmp_community": "",
      "snmp_oid": "",
      "hostid": "10336",
      "name": "System uptime",
      "key_": "system.uptime",
      "delay": "1m",
      "history": "1w",
      "trends": "365d",
      "status": "0",
      "value_type": "3",
      "trapper_hosts": "",
      "units": "uptime",
      "snmpv3_securityname": "",
      "snmpv3_securitylevel": "0",
      "snmpv3_authpassphrase": "",
      "snmpv3_privpassphrase": "",
      "formula": "",
      "error": "",
      "lastlogsize": "0",
      "logtimefmt": "",
      "templateid": "23150",
      "valuemapid": "0",
      "params": "",
      "ipmi_sensor": "",
      "authtype": "0",
      "username": "",
      "password": "",
      "publickey": "",
      "privatekey": "",
      "mtime": "0",
      "flags": "0",
      "interfaceid": "219",
      "port": "",
      "description": "",
      "inventory_link": "0",
      "lifetime": "30d",
      "snmpv3_authprotocol": "0",
      "snmpv3_privprotocol": "0",
      "state": "0",
      "snmpv3_contextname": "",
      "evaltype": "0",
      "jmx_endpoint": "",
      "master_itemid": "0",
      "lastclock": "1537556736",
      "lastns": "842421756",
      "lastvalue": "13830471",
      "prevvalue": "13830411"
    }
  ],
  "id": 1
}

From this values you can check important keys:
itemid
interfaceid
and many others

Now you can customize and create your new item.


Create new item (with using SNMP)


Create json file  item.json:
 
{
"jsonrpc": "2.0",
"method": "item.create",
"params": {
"name": "Active_Connection",
"key_": "Active_Connection",
"hostid": "10336",
"delay": "60s",
"history": "90d",
"type": 1,
"trends": "365d",
"status": "0",
"value_type": "3",
"snmp_oid": "1.3.6.1.4.1.140.625.190.1.25.16.18.144.146.180.197.194.18.77.60.51.78.5.246.62.52.18",
"snmp_community": "public",
"interfaceid": "219",
"port": "1161"
},
"auth": "e8a754b6fb4c55932bc6205dac3af55a",
"id": 0
}

And run


curl -i -X POST -H 'Content-type:application/json' -d @item.json http://zabbix/zabbix/api_jsonrpc.php


Friday, 14 September 2018

Zabbix monitoring Oracle database using orabbix - working example Oracle 12c

Orabbix


Orabix use jdbc connection to database so it not require installation of additional packages on database server.

Unzip downloaded Orabbix 1.2.3 file to /opt/orabbix (on zabbix server)

Copy file /opt/orabbix/init.d/orabbix to /etc/init.d/orabbix

Grant execute permissions to the following files:

 /etc/init.d/orabbix
 /opt/orabbix/run.sh


Create zabbix user on oracle database

CREATE USER ZABBIX
IDENTIFIED BY <REPLACE WITH PASSWORD>
DEFAULT TABLESPACE USERS
TEMPORARY TABLESPACE TEMP
PROFILE DEFAULT
ACCOUNT UNLOCK;
GRANT ALTER SESSION TO ZABBIX;
GRANT CREATE SESSION TO ZABBIX;
GRANT CONNECT TO ZABBIX;
ALTER USER ZABBIX DEFAULT ROLE ALL;
GRANT SELECT ON V_$INSTANCE TO ZABBIX;
GRANT SELECT ON DBA_USERS TO ZABBIX;
GRANT SELECT ON V_$LOG_HISTORY TO ZABBIX;
GRANT SELECT ON V_$PARAMETER TO ZABBIX;
GRANT SELECT ON SYS.DBA_AUDIT_SESSION TO ZABBIX;
GRANT SELECT ON V_$LOCK TO ZABBIX;
GRANT SELECT ON DBA_REGISTRY TO ZABBIX;
GRANT SELECT ON V_$LIBRARYCACHE TO ZABBIX;
GRANT SELECT ON V_$SYSSTAT TO ZABBIX;
GRANT SELECT ON V_$PARAMETER TO ZABBIX;
GRANT SELECT ON V_$LATCH TO ZABBIX;
GRANT SELECT ON V_$PGASTAT TO ZABBIX;
GRANT SELECT ON V_$SGASTAT TO ZABBIX;
GRANT SELECT ON V_$LIBRARYCACHE TO ZABBIX;
GRANT SELECT ON V_$PROCESS TO ZABBIX;
GRANT SELECT ON DBA_DATA_FILES TO ZABBIX;
GRANT SELECT ON DBA_TEMP_FILES TO ZABBIX;
GRANT SELECT ON DBA_FREE_SPACE TO ZABBIX;
GRANT SELECT ON V_$SYSTEM_EVENT TO ZABBIX;


 exec dbms_network_acl_admin.create_acl(acl => 'resolve.xml',description => 'resolve acl', principal =>'ZABBIX', is_grant => true, privilege => 'resolve');
 exec dbms_network_acl_admin.assign_acl(acl => 'resolve.xml', host =>'*');
 commit;


To verify connect to database as user zabbix and run sql

sqlplus zabbix/<REPLACE WITH PASSWORD>@database_service
select utl_inaddr.get_host_name('127.0.0.1') from dual;

(on RHEL6)
chkconfig -add orabbix



(on Debian) change line in file
/etc/init.d/orabbix 
from 
. /etc/rc.d/init.d/functions
to
. /lib/lsb/init-functions
 

Download from Oracle latest ojdbc6 driver and copy it to location /opt/orabbix/lib (overwrite existig file)


Example config.props (my db is rr01dev, zabbix server ip 192.168.41.194,
#comma separed list of Zabbix servers
ZabbixServerList=ZabbixServer1

ZabbixServer1.Address=192.168.41.194
ZabbixServer1.Port=10051

#pidFile
OrabbixDaemon.PidFile=./logs/orabbix.pid
#frequency of item's refresh
OrabbixDaemon.Sleep=300
#MaxThreadNumber should be >= than the number of your databases
OrabbixDaemon.MaxThreadNumber=100

#put here your databases in a comma separated list
DatabaseList=rr01dev

#Configuration of Connection pool
#if not specified Orabbis is going to use default values (hardcoded)
#Maximum number of active connection inside pool
DatabaseList.MaxActive=10
#The maximum number of milliseconds that the pool will wait
#(when there are no available connections) for a connection to be returned
#before throwing an exception, or <= 0 to wait indefinitely.
DatabaseList.MaxWait=100
DatabaseList.MaxIdle=1

#define here your connection string for each database
rr01dev.Url=jdbc:oracle:thin:@sc-test-001:1524/rr01dev
rr01dev.User=zabbix
rr01dev.Password=*****
#Those values are optionals if not specified Orabbix is going to use the general values
rr01dev.MaxActive=10
rr01dev.MaxWait=100
rr01dev.MaxIdle=1
rr01dev.QueryListFile=./conf/query.props



Database name must be the same as host in zabbix

Run script
/opt/orabbix/run.sh

And check for errors file
/opt/orabbix/logs/orabbix.log

When you have error with connection check if ListenPort=10051 is uncomented
/etc/zabbix/zabbix_server.conf
ListenPort=10051


Import templates from /opt/orabbix/templates to zabbix using gui

And create host

Apply template
Go to graph find your host and check if orabbix working properly

Tested on zabbix 3.4






Monday, 11 December 2017

Ansible - multiple if else statement or case alternative

Ansible prefer to use when: statement and not have classic implementation of if else or case statement

To use many if statement you can use simple trick.
Example check many variables and depend of this value (if variable is different than empty string '') counting latest occurrence of "not null" value


- name: count number of nic
  vars:
    count_nic: "{{ 5 if NETWORK_5 != '' else '4' if NETWORK_4 != '' else '3' if NETWORK_3 != '' else '2' if NETWORK_2 != '' else '1'}}"
#import playbook based on variable
  include: tasks/nic_{{ count_nic }}.yml
  register: count_nic

Ansible - Restart server and wait until available

Since ansible 2.3 you can use wait_for_connection function to check when host will be available

Example:

- name: Reboot server
  shell: ( sleep 2 && shutdown -r now & )
  async: 1
  poll: 0
  ignore_errors: true

- name: Waiting for host {{ ansible_default_ipv4.address }} after reboot
  wait_for_connection:
    timeout: 300
    delay: 30


This example solve your problem :-)

Ansible "The destination directory (/etc) is not writable"

I've used ansible playbook:

- name: copy resolv.conf
  become: root
  become_method: sudo
  copy:
    src: "resolv.conf"
    dest: "/etc/resolv.conf"
    owner: root
    group: root
    mode: 0644
    force: yes


Error "The destination directory (/etc) is not writable" occur

The problem is with function become - this function require true or false as parameter

Working configuration is  (SOLVED)

- name: copy resolv.conf
  become: yes
  become_method: sudo
  copy:
    src: "resolv.conf"
    dest: "/etc/resolv.conf"
    owner: root
    group: root
    mode: 0644
    force: yes
roles/os_conf_dn

Thursday, 11 May 2017

IBM V7000 compression performance - dedicated card for compression

Configuration of storage:
IBM V7000 gen2+
10x SSD 4TB
Interface 10GBe (iSCSI)
Dedicated card for compression

Configuration of test VM
2CPU, 4GB RAM
Linux Debian on ESX 6
Datastore mapped to ESX

Better IOPS during read operation and only a little lower on write. CPU usage for compression 30%.

Without compression:
test: (g=0): rw=randrw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=64
fio-2.0.9
Starting 1 process
test: Laying out IO file(s) (1 file(s) / 40960MB)

test: (groupid=0, jobs=1): err= 0: pid=3987: Wed Mar 29 07:57:35 2017
  read : io=30723MB, bw=139125KB/s, iops=34781 , runt=226126msec
  write: io=10237MB, bw=46360KB/s, iops=11589 , runt=226126msec
  cpu          : usr=9.58%, sys=37.75%, ctx=709816, majf=0, minf=4
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued    : total=r=7864963/w=2620797/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
   READ: io=30723MB, aggrb=139125KB/s, minb=139125KB/s, maxb=139125KB/s, mint=226126msec, maxt=226126msec
  WRITE: io=10237MB, aggrb=46359KB/s, minb=46359KB/s, maxb=46359KB/s, mint=226126msec, maxt=226126msec

Disk stats (read/write):
  sdb: ios=7856666/2618202, merge=0/45, ticks=10471096/3310356, in_queue=13777900, util=100.00%
test read 100%
test: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=64
fio-2.0.9
Starting 1 process

test: (groupid=0, jobs=1): err= 0: pid=3991: Wed Mar 29 08:00:59 2017
  read : io=40960MB, bw=206382KB/s, iops=51595 , runt=203230msec
  cpu          : usr=9.88%, sys=38.44%, ctx=665551, majf=0, minf=68
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued    : total=r=10485760/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
   READ: io=40960MB, aggrb=206382KB/s, minb=206382KB/s, maxb=206382KB/s, mint=203230msec, maxt=203230msec

Disk stats (read/write):
  sdb: ios=10475310/2, merge=0/1, ticks=11981140/0, in_queue=12170624, util=100.00%
test write 100% 
test: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=64
fio-2.0.9
Starting 1 process

test: (groupid=0, jobs=1): err= 0: pid=3995: Wed Mar 29 08:05:59 2017
  write: io=40960MB, bw=139572KB/s, iops=34892 , runt=300512msec
  cpu          : usr=7.17%, sys=29.45%, ctx=546661, majf=0, minf=4
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued    : total=r=0/w=10485760/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=40960MB, aggrb=139571KB/s, minb=139571KB/s, maxb=139571KB/s, mint=300512msec, maxt=300512msec

Disk stats (read/write):
  sdb: ios=0/10480731, merge=0/60, ticks=0/17984528, in_queue=17979712, util=100.00%


With compression:

test: (g=0): rw=randrw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=64
fio-2.0.9
Starting 1 process
test: Laying out IO file(s) (1 file(s) / 40960MB)

test: (groupid=0, jobs=1): err= 0: pid=4021: Wed Mar 29 08:54:43 2017
  read : io=30719MB, bw=164117KB/s, iops=41029 , runt=191668msec
  write: io=10241MB, bw=54715KB/s, iops=13678 , runt=191668msec
  cpu          : usr=10.62%, sys=45.65%, ctx=419306, majf=0, minf=4
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued    : total=r=7863971/w=2621789/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
   READ: io=30719MB, aggrb=164116KB/s, minb=164116KB/s, maxb=164116KB/s, mint=191668msec, maxt=191668msec
  WRITE: io=10241MB, aggrb=54715KB/s, minb=54715KB/s, maxb=54715KB/s, mint=191668msec, maxt=191668msec

Disk stats (read/write):
  sdb: ios=7853399/2618338, merge=0/38, ticks=7420120/3239660, in_queue=10970028, util=100.00%
test read 100%
test: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=64
fio-2.0.9
Starting 1 process

test: (groupid=0, jobs=1): err= 0: pid=4024: Wed Mar 29 08:56:51 2017
  read : io=40960MB, bw=328339KB/s, iops=82084 , runt=127743msec
  cpu          : usr=13.15%, sys=60.54%, ctx=294145, majf=0, minf=68
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued    : total=r=10485760/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
   READ: io=40960MB, aggrb=328339KB/s, minb=328339KB/s, maxb=328339KB/s, mint=127743msec, maxt=127743msec


Disk stats (read/write):
  sdb: ios=10466773/3, merge=0/1, ticks=6400748/8700, in_queue=6405540, util=100.00%
test write 100% 
test: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=64
fio-2.0.9
Starting 1 process

test: (groupid=0, jobs=1): err= 0: pid=4028: Wed Mar 29 09:02:27 2017
  write: io=40960MB, bw=125056KB/s, iops=31264 , runt=335393msec
  cpu          : usr=6.91%, sys=31.93%, ctx=491626, majf=0, minf=4
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued    : total=r=0/w=10485760/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=40960MB, aggrb=125056KB/s, minb=125056KB/s, maxb=125056KB/s, mint=335393msec, maxt=335393msec 

IBM V7000 performance DRAID5 vs DRAID6 on SSD

Configuration of storage:
IBM V7000 gen2+
10x SSD 4TB
Interface 10GBe (iSCSI)

Configuration of test VM
2CPU, 4GB RAM
Linux Debian on ESX 6
Datastore mapped to ESX

Performance limited by disks, CPU usage under 40% on storage.

DRAID5 (8D + 1P + 1S)

test: (g=0): rw=randrw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=64
fio-2.0.9
Starting 1 process
test: Laying out IO file(s) (1 file(s) / 40960MB)

test: (groupid=0, jobs=1): err= 0: pid=3987: Wed Mar 29 07:57:35 2017
  read : io=30723MB, bw=139125KB/s, iops=34781 , runt=226126msec
  write: io=10237MB, bw=46360KB/s, iops=11589 , runt=226126msec
  cpu          : usr=9.58%, sys=37.75%, ctx=709816, majf=0, minf=4
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued    : total=r=7864963/w=2620797/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
   READ: io=30723MB, aggrb=139125KB/s, minb=139125KB/s, maxb=139125KB/s, mint=226126msec, maxt=226126msec
  WRITE: io=10237MB, aggrb=46359KB/s, minb=46359KB/s, maxb=46359KB/s, mint=226126msec, maxt=226126msec

Disk stats (read/write):
  sdb: ios=7856666/2618202, merge=0/45, ticks=10471096/3310356, in_queue=13777900, util=100.00%
test read 100%
test: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=64
fio-2.0.9
Starting 1 process

test: (groupid=0, jobs=1): err= 0: pid=3991: Wed Mar 29 08:00:59 2017
  read : io=40960MB, bw=206382KB/s, iops=51595 , runt=203230msec
  cpu          : usr=9.88%, sys=38.44%, ctx=665551, majf=0, minf=68
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued    : total=r=10485760/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
   READ: io=40960MB, aggrb=206382KB/s, minb=206382KB/s, maxb=206382KB/s, mint=203230msec, maxt=203230msec

Disk stats (read/write):
  sdb: ios=10475310/2, merge=0/1, ticks=11981140/0, in_queue=12170624, util=100.00%
test write 100% 
test: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=64
fio-2.0.9
Starting 1 process

test: (groupid=0, jobs=1): err= 0: pid=3995: Wed Mar 29 08:05:59 2017
  write: io=40960MB, bw=139572KB/s, iops=34892 , runt=300512msec
  cpu          : usr=7.17%, sys=29.45%, ctx=546661, majf=0, minf=4
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued    : total=r=0/w=10485760/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=40960MB, aggrb=139571KB/s, minb=139571KB/s, maxb=139571KB/s, mint=300512msec, maxt=300512msec

Disk stats (read/write):
  sdb: ios=0/10480731, merge=0/60, ticks=0/17984528, in_queue=17979712, util=100.00%


DRAID6 (7D + 2P + 1S)
test: (g=0): rw=randrw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=64
fio-2.0.9
Starting 1 process
test: Laying out IO file(s) (1 file(s) / 40960MB)

test: (groupid=0, jobs=1): err= 0: pid=4021: Tue Mar 28 19:43:26 2017
  read : io=30719MB, bw=127238KB/s, iops=31809 , runt=247222msec
  write: io=10241MB, bw=42419KB/s, iops=10604 , runt=247222msec
  cpu          : usr=9.06%, sys=33.66%, ctx=787004, majf=0, minf=4
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued    : total=r=7864034/w=2621726/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
   READ: io=30719MB, aggrb=127238KB/s, minb=127238KB/s, maxb=127238KB/s, mint=247222msec, maxt=247222msec
  WRITE: io=10241MB, aggrb=42418KB/s, minb=42418KB/s, maxb=42418KB/s, mint=247222msec, maxt=247222msec

Disk stats (read/write):
  sdb: ios=7862987/2621478, merge=0/49, ticks=11209100/3770780, in_queue=14975764, util=100.00%
test read 100%
test: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=64
fio-2.0.9
Starting 1 process

test: (groupid=0, jobs=1): err= 0: pid=4025: Tue Mar 28 19:47:06 2017
  read : io=40960MB, bw=190892KB/s, iops=47723 , runt=219721msec
  cpu          : usr=9.46%, sys=35.56%, ctx=756041, majf=0, minf=68
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued    : total=r=10485760/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
   READ: io=40960MB, aggrb=190892KB/s, minb=190892KB/s, maxb=190892KB/s, mint=219721msec, maxt=219721msec

Disk stats (read/write):
  sdb: ios=10478253/3, merge=0/1, ticks=13149764/169304, in_queue=13315696, util=100.00%
test write 100% 
test: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=64
fio-2.0.9
Starting 1 process

test: (groupid=0, jobs=1): err= 0: pid=4029: Tue Mar 28 19:53:33 2017
  write: io=40960MB, bw=108597KB/s, iops=27149 , runt=386225msec
  cpu          : usr=5.93%, sys=22.56%, ctx=660059, majf=0, minf=4
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued    : total=r=0/w=10485760/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=40960MB, aggrb=108597KB/s, minb=108597KB/s, maxb=108597KB/s, mint=386225msec, maxt=386225msec

Disk stats (read/write):
  sdb: ios=0/10484791, merge=0/83, ticks=0/23609532, in_queue=23605968, util=100.00%

Monday, 29 December 2014

Configure bamboo with NEXUS - Sonatype

I use for configuration bamboo 5.7.1 and all configuration will be done from script - I not use plugins

First I prepare build project consists from 2 stages
- one for prepare package - for example zip file
- two for update package to repository - NEXUS

When I have zip file to upload to NEXUS I use NEXUS api to upload file
#use timestamp to versioning
VERSION=`echo "${bamboo.buildTimeStamp}" | sed 's/:/-/g' | cut -f 1 -d "."`
#uploadcurl -v \
   -F "r=releases" \ 
   -F "g=pl.company.sql" \ 
   -F "hasPom=false" \
   -F "a=db" \
   -F "c=${bamboo.buildResultKey}" \
   -F "p=zip" \    -F "v=$VERSION" \
   -F "file=@./$f" \
   -u  deployment:${bamboo.nexuspassword}\ 
   http://service/nexus/service/local/artifact/maven/content
important is use classifier

or You can use MAVEN

Second - download script for deployment project

#Fix mapping between buildResultKey
CLASS=`echo ${bamboo.buildResultKey} | sed 's/-/-RDP-/2'`
-For more information see post below and continue
#find package by classifier
VERSION=`curl -u  deployment:${bamboo.nexuspassword} "http://service/nexus/service/local/lucene/search?g=$GROUP&a=$ARTIFACT&c=$CLASS&r=$REPO" | sed -n 's|<version>\(.*\)</version>|\1|p' | sed -e 's/^[ \t]*//' -e 's/[ \t]*$//'`

#Download by wget
wget -O db-sprint.zip --user deployment --password=${bamboo.nexuspassword} "http://service/nexus/service/local/artifact/maven/content?r=$REPO&g=$GROUP&a=$ARTIFACT&v=$VERSION&p=zip&c=$CLASS"



Configure mapping between build project and deployment project in Bamboo

Atlassian Bamboo don't have formal mapping between build project and deployment project but you can use buildResultKey - it constain number of build which is constant.

I create build plan and deployment plan in deployment plan set to update buildResultKey to be correct for deployment plan

echo ${bamboo.buildResultKey} | sed 's/-/-JOB1-/2'

You shoud list buildResultKey variable in build project and deployment project and check difference - for me it is JOB1

Monday, 13 January 2014

Targetcli performance - 1Gbit network

Testing targetcli - cofigured in Targetcli configuration iSCSI

create ramdrive on target
mount -t tmpfs -o size=7G tmpfs /mnt/ramd
Create file in ramdrive
dd if=/dev/zero of=/mnt/ramd/test1 bs=1024 count=6000000
6000000+0 records in
6000000+0 records out
6144000000 bytes (6.1 GB) copied, 8.75343 s, 702 MB/s

Map file to datastore in targetcli
Connect to target
iscsiadm -m node --targetname "iqn.2003-01.org.linux-iscsi.sc-azl-001.x8664:sn.1a7b82ef561a" --portal "192.168.42.135:3260" --login
Logging in to [iface: default, target: iqn.2003-01.org.linux-iscsi.sc-azl-001.x8664:sn.1a7b82ef561a, portal: 192.168.42.135,3260]
Login to [iface: default, target: iqn.2003-01.org.linux-iscsi.sc-azl-001.x8664:sn.1a7b82ef561a, portal: 192.168.42.135,3260] successful.

dmesg | tail
[3658097.364033] sd 11:0:0:0: [sdc] Write Protect is off
[3658097.364033] sd 11:0:0:0: [sdc] Mode Sense: 43 00 00 08
[3658097.364559] sd 11:0:0:0: [sdc] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[3658097.365159] sd 11:0:0:0: [sdc] 12000000 512-byte hardware sectors (6144 MB)
[3658097.365463] sd 11:0:0:0: [sdc] Write Protect is off
[3658097.365463] sd 11:0:0:0: [sdc] Mode Sense: 43 00 00 08
[3658097.365945] sd 11:0:0:0: [sdc] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA


Create partition
fdisk /dev/sdc
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0xf5daec87.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
         switch off the mode (command 'c') and change display units to
         sectors (command 'u').

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1018, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-1018, default 1018):
Using default value 1018

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

Create filesystem and mount
mkfs.ext3 /dev/sdc1
mount /dev/sdc1 /mnt/istest

Write test on initiator
dd if=/dev/zero of=/mnt/istest/test1 bs=1024 count=5600000
5600000+0 records in
5600000+0 records out
5734400000 bytes (5.7 GB) copied, 46.0264 s, 125 MB/s

iSCSI reach network interface limit - 1Gbit
CPU usage on iSCSI target

top - 19:07:58 up 50 min,  1 user,  load average: 0.00, 0.01, 0.05
Tasks: 107 total,   2 running, 105 sleeping,   0 stopped,   0 zombie
%Cpu(s):  0.0 us,  7.5 sy,  0.0 ni, 92.4 id,  0.0 wa,  0.0 hi,  0.2 si,  0.0 st
KiB Mem:  10268436 total,  6240584 used,  4027852 free,     9448 buffers
KiB Swap:   561148 total,        0 used,   561148 free,  6138880 cached

  PID USER      PR  NI  VIRT  RES  SHR S  %CPU %MEM    TIME+  COMMAND
 3037 root     20   0   0    0    0    R    17.3  0.0   0:03.89 iscsi_trx
 3036 root     20   0   0    0    0    S     1.3  0.0   0:00.28 iscsi_ttx


Tested in virtual machine with 10GB RAM, 2 core (phisycal cpu xeon E5520)



After configuring multipath on client - initiator

sc-srv-006:~# dd if=/dev/zero of=/mnt/istest/test1 bs=1024 count=5600000
5600000+0 records in
5600000+0 records out
5734400000 bytes (5.7 GB) copied, 29.79 s, 192 MB/s

Network statistics from target iSCSI

root@sc-azl-001:~# ethstats
total:     0.01 Mb/s In     0.00 Mb/s Out -     15.0 p/s In       0.0 p/s Out
  eth0:    0.01 Mb/s In     0.00 Mb/s Out -      8.0 p/s In       0.0 p/s Out
  eth1:    0.01 Mb/s In     0.00 Mb/s Out -      7.0 p/s In       0.0 p/s Out
total:     2.04 Mb/s In     0.07 Mb/s Out -    244.9 p/s In      97.8 p/s Out
  eth0:    0.98 Mb/s In     0.04 Mb/s Out -    121.0 p/s In      47.3 p/s Out
  eth1:    1.06 Mb/s In     0.04 Mb/s Out -    123.9 p/s In      50.5 p/s Out
total:   288.21 Mb/s In     1.37 Mb/s Out -  24193.5 p/s In    2389.0 p/s Out
  eth0:  152.06 Mb/s In     0.67 Mb/s Out -  12790.5 p/s In    1154.3 p/s Out
  eth1:  136.15 Mb/s In     0.70 Mb/s Out -  11403.0 p/s In    1234.7 p/s Out
total:  1307.89 Mb/s In     6.22 Mb/s Out - 110060.5 p/s In   10903.6 p/s Out
  eth0:  666.02 Mb/s In     2.97 Mb/s Out -  56061.4 p/s In    5172.4 p/s Out
  eth1:  641.87 Mb/s In     3.25 Mb/s Out -  53999.1 p/s In    5731.2 p/s Out
total:  1286.38 Mb/s In     6.15 Mb/s Out - 108248.9 p/s In   10771.6 p/s Out
  eth0:  642.20 Mb/s In     2.88 Mb/s Out -  54058.7 p/s In    5011.2 p/s Out
  eth1:  644.18 Mb/s In     3.27 Mb/s Out -  54190.2 p/s In    5760.4 p/s Out
total:  1294.30 Mb/s In     6.19 Mb/s Out - 108921.4 p/s In   10854.0 p/s Out
  eth0:  648.03 Mb/s In     2.92 Mb/s Out -  54556.0 p/s In    5089.5 p/s Out
  eth1:  646.27 Mb/s In     3.27 Mb/s Out -  54365.4 p/s In    5764.5 p/s Out
total:   631.95 Mb/s In     3.05 Mb/s Out -  53211.1 p/s In    5316.6 p/s Out
  eth0:  296.08 Mb/s In     1.34 Mb/s Out -  24938.3 p/s In    2310.2 p/s Out

  eth1:  335.87 Mb/s In     1.71 Mb/s Out -  28272.8 p/s In    3006.4 p/s Out

FC target configuration - Targetcli as open source SAN (debian)

Installation on debian wheezy with kernel 3.10.25

Download rpm
apt-get install targetcli 

Or install without recommended packages
apt-get install --no-install-recommends targetcli

Create file for datastore
dd if=/dev/zero of=/opt/test1 bs=1024 count=6000000



targetcli

Create datastore on targetcli
cd backstores/fileio
create itest1 /opt/test1 6144000000B

Create iSCSCI target instance
cd /iscsi
create

Map LUN to datastore (based on FILEIO)
cd luns
create /backstores/fileio/itest1

Configure ACL (check WWN in openiscsi section)
cd ../../acls
create
create iqn.1993-08.org.debian:01:dcb3d359cc91

Save configuration
cd /
saveconfig

Disable athentication
cd iscsi/
set set discovery_auth enable=0
cd /iscsi/iqn.2003-01.org.linux-iscsi.sc-azl-001.x8664:sn.8b4b5e8c13a2/tpgt1
set attribute authentication=0

Configure interface
cd portals
create
cd /
saveconfig
All config should looks like:

targetcli
ls
o- / ......................................................................................................... [...]
  o- backstores .............................................................................................. [...]
  | o- fileio ................................................................................... [1 Storage Object]
  | | o- idisk ............................................................................. [/opt/test1 activated]
  | o- iblock ................................................................................... [0 Storage Object]
  | o- pscsi .................................................................................... [0 Storage Object]
  | o- rd_dr .................................................................................... [0 Storage Object]
  | o- rd_mcp ................................................................................... [0 Storage Object]
  o- iscsi .............................................................................................. [1 Target]
  | o- iqn.2003-01.org.linux-iscsi.sc-azl-001.x8664:sn.8b4b5e8c13a2 ........................................ [1 TPG]
  |   o- tpgt1 ........................................................................................... [enabled]
  |     o- acls ........................................................................................... [1 ACLs]
  |     | o- iqn.1993-08.org.debian:01:dcb3d359cc91 ................................................. [1 Mapped LUN]
  |     |   o- mapped_lun0 ............................................................................. [lun0 (rw)]
  |     o- luns ............................................................................................ [1 LUN]
  |     | o- lun0 ..................................................................... [fileio/idisk (/home/test1)]
  |     o- portals ...................................................................................... [1 Portal]
  |       o- 192.168.42.135:3260 .............................................................................. [OK]
  o- loopback ........................................................................................... [0 Target]
  o- qla2xxx ............................................................................................ [0 Target]
  o- tcm_fc ............................................................................................. [0 Target]


OpeniSCSI installation - client (initiatior)

aptitude install open-iscsi
/etc/init.d/open-iscsi restart

Search target name:
iscsiadm -m discovery -t sendtargets -p 192.168.42.135

Check your initiator name

cat /etc/iscsi/initiatorname.iscsi
## DO NOT EDIT OR REMOVE THIS FILE!
## If you remove this file, the iSCSI daemon will not start.
## If you change the InitiatorName, existing access control lists
## may reject this initiator.  The InitiatorName must be unique
## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.
InitiatorName=iqn.1993-08.org.debian:01:dcb3d359cc91



iscsiadm -m node

Connect to iscsi target:
iscsiadm -m node --targetname "iqn.2003-01.org.linux-iscsi.sc-azl-001.x8664:sn.8b4b5e8c13a2" --portal "192.168.42.135:3260" --login

Verify new block device
dmesg | tail

Debian wheezy kernel upgrade to 3.10 installation

Install packages needed to compile:

apt-get install fakeroot build-essential ncurses-dev

Download and unpack kernel

wget https://www.kernel.org/pub/linux/kernel/v3.x/linux-3.10.25.tar.bz2
tar -xvf linux-3.10.25.tar.bz2
cd linux-3.10.25/

Configure kernel base on old configuration
cat /boot/config-`uname -r` > .config
cat .config
make oldconfig
make menuconfig

Compile kernel

time make deb-pkg -j2

Install
dpkg -i ../linux-firmware-image_3.10.25-1_amd64.deb ../linux-headers-3.10.25_3.10.25-1_amd64.deb ../linux-libc-dev_3.10.25-1_amd64.deb ../linux-image-3.10.25_3.10.25-1_amd64.deb
apt-cache policy linux-libc-dev

restart debian

Tuesday, 29 October 2013

Convert logs to cvs with exclude some columns

Param (

$FILE_IN="access.log",
$FILE_OUT="access.csv",
$FILE_TMP="tmp.csv"
)

$datetime = get-date -f yyddMM-hhmm

cat $FILE_IN | where { $_ -match "^[0-9]"} | % { $_ -replace '\s+',"," } | foreach {Write-output "$(($_ -split ',')[1,4,5])" } | % { $_ -replace '\s+',"," } | Out-File -append -filePath $FILE_TMP

$FILE_OUT=$FILE_OUT.replace("csv",$datetime + ".csv")
Write-output $FILE_OUT
Copy-Item $FILE_TMP $FILE_OUT


Tuesday, 15 January 2013

RMAN-06059: expected archived log not found

If rman unable to find archivelog run crosscheck


change archivelog all crosscheck;


Monday, 31 December 2012

Tuesday, 18 December 2012

Recreate control file error ORA-01194:

How to recreate controlfile

http://oracleinstance.blogspot.com/2010/09/re-create-lost-controlfile.html


Monday, 5 November 2012

Weblogic clone/ change ip of manage server



BEA-090482 - BAD_CERTIFICATE


1. Disable Flags
Put the following flags at the right places.
Node Manager: -Dweblogic.nodemanager.sslHostNameVerificationEnabled=false
Admin Server: -Dweblogic.security.SSL.ignoreHostnameVerification=true

2. Recreate the Certificates - The recommended way.
Node manager by default uses the WebLogic demo identity keystore. The keystore is generated at install time using the CertGen utility. The generated private key uses the common name (cn) resolved by Java.

2.1 Set the PATH
. $WL_HOME/server/bin/setWLSEnv.sh

2.2 Backup DemoIdentity.jks under $WL_HOME/server/lib

2.3 Generate the private key.
java utils.CertGen -cn -keyfilepass DemoIdentityPassPhrase -certfile newcert -keyfile newkey

2.4 Import the key generated above to the keystore.
java utils.ImportPrivateKey -keystore DemoIdentity.jks -storepass DemoIdentityKeyStorePassPhrase -keyfile newkey.pem -keyfilepass DemoIdentityPassPhrase -certfile newcert.pem -alias demoidentity

2.5 Copy DemoIdentity.jks to $WL_HOME/server/lib

2.6 Restart your nodemanager.

Tuesday, 25 September 2012

Loop xargs

$ find . -iname "*.mp3" -print0 | xargs -0 -I mp3file mplayer mp3file

Connect to iscsi disk debian

List device
iscsiadm -m discovery -t st -p 192.168.42.236

192.168.42.236:3260,1 iqn.2012-08.sc-nas-004.istgt:disk1

Connect:
iscsiadm -m discovery -t st -p 192.168.42.236 "iqn.2012-08.sc-nas-004.istgt:disk2" --portal "192.168.42.236:3260" --login

Monday, 24 September 2012

Oracle EM repository recreate

 emca -repos recreate

emca -config dbcontrol db
change ports

emca -reconfig ports -dbcontrol_http_port 1158