Program runs very slow on a Oracle database server of Linux 64-bit -
i have program takes in average 30 minutes run on general 64-bit linux machines. takes 150 minutes run on 64-bit linux machine oracle database running.
vmstat shows program need in average 6gb memory , bi=0 on general machines. uses no more 2gb memory on oracle database server , bi high.
the oracle database server has total memory=500g, sga=42112m , pga=20g.
my question oracle setup restrict memory usage of other program.
the following little part of top output when program running:
top - 15:34:08 114 days, 12:27, 11 users, load average: 0.48, 0.60, 0.54 tasks: 872 total, 2 running, 870 sleeping, 0 stopped, 0 zombie cpu(s): 15.8%us, 3.2%sy, 0.0%ni, 64.6%id, 16.3%wa, 0.0%hi, 0.2%si, 0.0%st mem: 529409524k total, 523431316k used, 5978208k free, 788964k buffers swap: 20971516k total, 247040k used, 20724476k free, 234761252k cached pid user pr ni virt res shr s %cpu %mem time+ command 49096 oracle 20 0 41.5g 1.1g 1.0g s 47.5 0.2 0:14.90 oracle 17862 oracle 20 0 41.5g 46m 32m r 18.3 0.0 5:09.58 oracle 17874 oracle 20 0 41.5g 47m 31m s 16.5 0.0 4:50.99 oracle 35934 root rt 0 769m 101m 55m s 12.8 0.0 5830:15 osysmond.bin 17864 oracle 20 0 41.5g 46m 31m s 11.0 0.0 4:53.54 oracle 17882 oracle 20 0 41.5g 46m 31m s 7.3 0.0 5:20.39 oracle 52363 applmgr 20 0 15636 1760 828 r 7.3 0.0 0:00.08 top 35064 root 20 0 1974m 47m 14m s 3.7 0.0 642:05.39 ohasd.bin 26123 oracle 20 0 33.4g 33m 16m s 1.8 0.0 74:18.77 oracle 27764 oracle 20 0 33.4g 21m 19m s 1.8 0.0 0:41.09 oracle 36387 root 20 0 814m 20m 10m s 1.8 0.0 384:21.37 octssd.bin 38011 oracle -2 0 1237m 14m 12m s 1.8 0.0 1212:52 oracle 38130 root 20 0 2910m 60m 20m s 1.8 0.0 1070:16 crsd.bin 38192 oracle 20 0 1239m 17m 15m s 1.8 0.0 21:21.70 oracle 39657 root 20 0 1582m 23m 11m s 1.8 0.0 1108:05 orarootagent.bi 41587 oracle 20 0 33.4g 23m 20m s 1.8 0.0 0:00.14 oracle 49809 oracle 20 0 41.4g 57m 54m s 1.8 0.0 0:00.73 oracle 1 root 20 0 21460 1564 1240 s 0.0 0.0 44:37.17 init 2 root 20 0 0 0 0 s 0.0 0.0 1:05.13 kthreadd 3 root 20 0 0 0 0 s 0.0 0.0 18:12.77 ksoftirqd/0 6 root rt 0 0 0 0 s 0.0 0.0 163191:59 migration/0 7 root rt 0 0 0 0 s 0.0 0.0 0:50.35 watchdog/0 8 root rt 0 0 0 0 s 0.0 0.0 164526:35 migration/1 10 root 20 0 0 0 0 s 0.0 0.0 18:48.93 ksoftirqd/1 12 root rt 0 0 0 0 s 0.0 0.0 0:51.10 watchdog/1 13 root rt 0 0 0 0 s 0.0 0.0 161719:04 migration/2 15 root 20 0 0 0 0 s 0.0 0.0 8:58.82 ksoftirqd/2 16 root rt 0 0 0 0 s 0.0 0.0 1:00.06 watchdog/2 17 root rt 0 0 0 0 s 0.0 0.0 162585:32 migration/3 19 root 20 0 0 0 0 s 0.0 0.0 6:18.01 ksoftirqd/3 20 root rt 0 0 0 0 s 0.0 0.0 0:38.44 watchdog/3 21 root rt 0 0 0 0 s 0.0 0.0 158088:12 migration/4 23 root 20 0 0 0 0 s 0.0 0.0 12:46.60 ksoftirqd/4 24 root rt 0 0 0 0 s 0.0 0.0 0:44.32 watchdog/4 25 root rt 0 0 0 0 s 0.0 0.0 164162:22 migration/5 27 root 20 0 0 0 0 s 0.0 0.0 14:00.94 ksoftirqd/5 28 root rt 0 0 0 0 s 0.0 0.0 0:39.26 watchdog/5 29 root rt 0 0 0 0 s 0.0 0.0 162112:29 migration/6 31 root 20 0 0 0 0 s 0.0 0.0 7:36.89 ksoftirqd/6 32 root rt 0 0 0 0 s 0.0 0.0 0:36.18 watchdog/6 33 root rt 0 0 0 0 s 0.0 0.0 160716:53 migration/7 35 root 20 0 0 0 0 s 0.0 0.0 3:33.29 ksoftirqd/7 36 root rt 0 0 0 0 s 0.0 0.0 0:32.82 watchdog/7 37 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 cpuset 38 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 khelper 39 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 netns 40 root 20 0 0 0 0 s 0.0 0.0 0:38.28 sync_supers 41 root 20 0 0 0 0 s 0.0 0.0 0:06.53 bdi-default 42 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kintegrityd 43 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kblockd 44 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 ata_sff 45 root 20 0 0 0 0 s 0.0 0.0 0:00.01 khubd 46 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 md 50 root 20 0 0 0 0 s 0.0 0.0 0:30.08 khungtaskd 51 root 20 0 0 0 0 s 0.0 0.0 13:30.48 kswapd0 52 root 25 5 0 0 0 s 0.0 0.0 0:00.00 ksmd 53 root 20 0 0 0 0 s 0.0 0.0 0:00.03 fsnotify_mark 54 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 crypto 60 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kthrotld 62 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kpsmoused 349 root 20 0 0 0 0 s 0.0 0.0 0:00.00 scsi_eh_0 351 root 20 0 0 0 0 s 0.0 0.0 0:14.72 hpsa 364 oracle 20 0 33.4g 37m 31m s 0.0 0.0 0:14.78 oracle 366 oracle 20 0 33.4g 34m 28m s 0.0 0.0 0:04.32 oracle 374 oracle 20 0 33.4g 37m 30m s 0.0 0.0 0:05.47 oracle 377 root 20 0 0 0 0 s 0.0 0.0 0:00.05 scsi_eh_1 378 root 20 0 0 0 0 s 0.0 0.0 0:00.01 scsi_eh_2 402 root 20 0 102m 4024 3052 s 0.0 0.0 0:00.26 sshd 484 root 20 0 36268 1340 1048 s 0.0 0.0 0:00.44 vncconfig 485 root 20 0 150m 3916 2220 s 0.0 0.0 0:00.75 xterm 486 root 20 0 127m 2184 1484 s 0.0 0.0 0:00.52 twm 493 root 20 0 0 0 0 s 0.0 0.0 0:00.00 scsi_eh_3 494 root 20 0 0 0 0 s 0.0 0.0 0:00.76 usb-storage 497 root 20 0 107m 1860 1460 s 0.0 0.0 0:00.08 bash 537 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 539 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 558 root 20 0 0 0 0 s 0.0 0.0 0:04.83 jbd2/dm-1-8 559 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 ext4-dio-unwrit 633 root 20 0 0 0 0 s 0.0 0.0 0:15.35 kworker/3:0 665 root 16 -4 17568 7752 460 s 0.0 0.0 22631:49 udevd 673 oracle 20 0 102m 2004 1032 s 0.0 0.0 0:00.05 sshd 697 oracle 20 0 108m 2048 1444 s 0.0 0.0 0:00.06 bash 771 oracle 20 0 33.4g 25m 21m s 0.0 0.0 0:30.99 oracle 792 root 20 0 0 0 0 s 0.0 0.0 0:00.00 scsi_eh_4 1017 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 qla2xxx_4_dpc 1018 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 scsi_wq_4 1021 root 20 0 0 0 0 s 0.0 0.0 0:00.00 scsi_eh_5 1037 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 qla2xxx_5_dpc 1038 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 scsi_wq_5 1075 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 qlcnic 1135 hpsmh 20 0 176m 11m 4380 s 0.0 0.0 0:25.39 hpsmhd 1677 oracle 20 0 41.4g 118m 113m s 0.0 0.0 0:01.45 oracle 1798 oracle 20 0 41.4g 56m 51m s 0.0 0.0 0:00.59 oracle 1974 root 20 0 0 0 0 s 0.0 0.0 0:00.02 flush-252:109 1987 root 20 0 102m 4060 3092 s 0.0 0.0 0:00.29 sshd 1993 sysmon 20 0 102m 2004 1036 s 0.0 0.0 0:00.78 sshd 1994 sysmon 20 0 105m 1888 1440 s 0.0 0.0 0:00.73 bash 2084 root 20 0 158m 4072 2592 s 0.0 0.0 0:00.08 sudo 2110 root 20 0 107m 1580 1236 s 0.0 0.0 0:00.01 sh 2488 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kmpathd 2489 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kmpath_handlerd 2606 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2612 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2616 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2620 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2624 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2628 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2635 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2638 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2645 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2652 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2658 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2662 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2667 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2672 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2676 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2683 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2686 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2687 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2692 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2699 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2708 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2715 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2718 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2720 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2728 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2734 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2743 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2750 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2753 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2755 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2761 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2766 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2774 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2777 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2782 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2784 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2787 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2790 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2801 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2812 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2814 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2820 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2823 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2827 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2833 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2837 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2842 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2848 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2854 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2864 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2877 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2888 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2901 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2905 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2912 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2920 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2922 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2931 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2940 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2952 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2953 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2955 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2972 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2977 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2983 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2989 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2990 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 2996 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 3003 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 3010 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 3014 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 3021 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 3027 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 3033 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 3046 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 3053 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 3064 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 3072 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 3092 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 3097 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 3108 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 3114 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 3119 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 3126 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 3133 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 3139 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 3148 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 3153 oracle 20 0 4560m 304m 12m s 0.0 0.1 157:16.04 java 3154 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 3162 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 3397 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 3399 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 3427 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 3491 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 3492 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 3494 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 3496 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 3499 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 3502 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 3506 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 3511 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 3520 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 3523 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 3529 root 0 -20 0 0 0 s 0.0 0.0 0:00.00 kdmflush 3613 root 20 0 0 0 0 s 0.0 0.0 0:00.02 kjournald
we have not enough information determine happening in server. ways know oracle setup can affect process behavior (without considering ram/cpu/disk overload) are:
- ulimit: oracle setup uses command limit maximum number of open file descriptors , limit maximum number of process per user. command can used limit max amount of resident memory per process, oracle not use in way.
- kernel shared memory limit: oracle setup defines shmmax , shmall , these parameters limit size of shared memory available in linux kernel.
but these limits affects memory consumption of program if:
- it working many opened files simultaneously (then affected ulimit)
- it spawning processes (to perform parallel tasks) (then affected ulimit)
- it allocating , using shared memory (the same oracle uses sga) (then affected shmmax , shmall).
there other linux kernel parameters, process , services can affect maximum size of memory resident available process (the popular ones are: via limits.conf, via upstart , via cgroup – see https://afrobeard.wordpress.com/2012/02/01/linux-limiting-resident-memory/). ways not derived oracle setup, derived kernel configuration. must check them.
note: determine if limit in ram usage derived kernel configuration can create program (using java, c, c++ or language want) reserves , use 6g (e.g. creating arrays of arrays , setting values) , test if can use more 2g.
to determine if oracle workload interferes program can try test program in oracle server database instance down (if possible).
Comments
Post a Comment