93i386 dumped core - see /var/crash/vmcore.0 Wed Apr 1 18:28:16 CEST 2015 FreeBSD 93i386 9.3-RELEASE-p10 FreeBSD 9.3-RELEASE-p10 #0: Tue Feb 24 21:01:19 UTC 2015 root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC i386 panic: kmem_malloc(86016): kmem_map too small: 431890432 total allocated GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-marcel-freebsd"... Unread portion of the kernel message buffer: panic: kmem_malloc(86016): kmem_map too small: 431890432 total allocated cpuid = 2 KDB: stack backtrace: #0 0xc0b0f96f at kdb_backtrace+0x4f #1 0xc0ad65af at panic+0x16f #2 0xc0d62c7a at kmem_malloc+0x28a #3 0xc0d56387 at page_alloc+0x27 #4 0xc0d58a30 at uma_large_malloc+0x50 #5 0xc0abdf3c at malloc+0x8c #6 0xc86caf50 at zfs_kmem_alloc+0x20 #7 0xc860dd34 at zio_buf_alloc+0x44 #8 0xc85ef8cd at vdev_queue_io_to_issue+0x95d #9 0xc85efbbc at vdev_queue_io_done+0xec #10 0xc860d511 at zio_vdev_io_done+0x81 #11 0xc860da4e at zio_execute+0xee #12 0xc0b1c6f2 at taskqueue_run_locked+0xd2 #13 0xc0b1d4dc at taskqueue_thread_loop+0xbc #14 0xc0aa243f at fork_exit+0xcf #15 0xc0f88734 at fork_trampoline+0x8 Uptime: 32m19s Physical memory: 3499 MB Dumping 537 MB: 522 506 490 474 458 442 426 410 394 378 362 346 330 314 298 282 266 250 234 218 202 186 170 154 138 122 106 90 74 58 42 26 10 Reading symbols from /boot/kernel/zfs.ko...Reading symbols from /boot/kernel/zfs.ko.symbols...done. done. Loaded symbols for /boot/kernel/zfs.ko Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from /boot/kernel/opensolaris.ko.symbols...done. done. Loaded symbols for /boot/kernel/opensolaris.ko #0 doadump (textdump=1) at pcpu.h:250 250 pcpu.h: No such file or directory. in pcpu.h (kgdb) #0 doadump (textdump=1) at pcpu.h:250 #1 0xc0ad62f5 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:454 #2 0xc0ad65f2 in panic (fmt=) at /usr/src/sys/kern/kern_shutdown.c:642 #3 0xc0d62c7a in kmem_malloc (map=0xc1bdd090, size=86016, flags=2) at /usr/src/sys/vm/vm_kern.c:305 #4 0xc0d56387 in page_alloc (zone=0x0, bytes=86016, pflag=0xf0de97ef "\002", wait=2) at /usr/src/sys/vm/uma_core.c:994 #5 0xc0d58a30 in uma_large_malloc (size=86016, wait=2) at /usr/src/sys/vm/uma_core.c:3071 #6 0xc0abdf3c in malloc (size=86016, mtp=0xc86cc11c, flags=2) at /usr/src/sys/kern/kern_malloc.c:527 #7 0xc86caf50 in zfs_kmem_alloc (size=82944, kmflags=2) at /usr/src/sys/modules/opensolaris/../../cddl/compat/opensolaris/kern/opensolaris_kmem.c:74 #8 0xc860dd34 in zio_buf_alloc (size=82944) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:267 #9 0xc85ef8cd in vdev_queue_io_to_issue (vq=0xc989cb00) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:623 #10 0xc85efbbc in vdev_queue_io_done (zio=0xe15902dc) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:773 #11 0xc860d511 in zio_vdev_io_done (ziop=0xf0de9c6c) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:2644 #12 0xc860da4e in zio_execute (zio=0xe15902dc) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1341 #13 0xc0b1c6f2 in taskqueue_run_locked (queue=0xc9bb4000) at /usr/src/sys/kern/subr_taskqueue.c:321 #14 0xc0b1d4dc in taskqueue_thread_loop (arg=0xc8b8fa50) at /usr/src/sys/kern/subr_taskqueue.c:529 #15 0xc0aa243f in fork_exit (callout=0xc0b1d420 , arg=0xc8b8fa50, frame=0xf0de9d08) at /usr/src/sys/kern/kern_fork.c:996 #16 0xc0f88734 in fork_trampoline () at /usr/src/sys/i386/i386/exception.s:279 (kgdb) ------------------------------------------------------------------------ ps -axlww UID PID PPID CPU PRI NI VSZ RSS MWCHAN STAT TT TIME COMMAND 0 0 0 0 -8 0 0 0 - DLs ?? 0:01.51 [kernel] 0 1 0 0 20 0 8036 2638720 wait DLs ?? 0:00.01 [init] 0 2 0 0 -16 0 0 0 waiting_ DL ?? 0:00.00 [sctp_iterator] 0 3 0 0 -16 0 0 0 - DL ?? 0:00.00 [xpt_thrd] 0 4 0 0 -16 0 0 0 psleep DL ?? 0:00.00 [pagedaemon] 0 5 0 0 -16 0 0 0 psleep DL ?? 0:00.00 [vmdaemon] 0 6 0 0 155 0 0 0 pgzero DL ?? 0:00.00 [pagezero] 0 7 0 0 -16 0 0 0 psleep DL ?? 0:00.00 [bufdaemon] 0 8 0 0 16 0 0 0 syncer DL ?? 0:00.01 [syncer] 0 9 0 0 -16 0 0 0 vlruwt DL ?? 0:00.00 [vnlru] 0 10 0 0 -16 0 0 0 audit_wo DL ?? 0:00.00 [audit] 0 11 0 0 155 0 0 0 - RL ?? 6:52.59 [idle] 0 12 0 0 -88 0 0 0 - WL ?? 0:00.81 [intr] 0 13 0 0 -8 0 0 0 - DL ?? 0:00.63 [geom] 0 14 0 0 -16 0 0 0 - DL ?? 0:00.01 [yarrow] 0 15 0 0 -16 0 0 0 sdflush DL ?? 0:00.00 [softdepflush] 0 33 0 0 -8 0 0 0 zio->io_ DL ?? 0:00.06 [zfskern] 0 131 0 0 -8 0 0 0 mdwait DL ?? 0:00.00 [md0] 0 312 1 0 52 0 9552 401664 select Ds ?? 0:00.00 [dhclient] 65 350 1 0 24 0 9552 2547024 select Ds ?? 0:00.00 [dhclient] 0 363 1 0 20 0 12132 4000480 select Ds ?? 0:00.00 [devd] 0 452 1 0 20 0 9552 275312 select Ds ?? 0:00.01 [syslogd] 0 537 1 0 20 0 9552 1478288 sbwait D ?? 0:00.00 [sempty] 0 538 537 0 47 0 9860 279424 wait Ds+ ?? 0:00.01 [sh] 0 579 1 0 20 0 11260 1463792 select Ds ?? 0:00.01 [ntpd] 0 593 1 0 21 0 10548 2667376 select Ds ?? 0:00.00 [rsync] 0 624 538 0 52 0 9860 135744 wait D+ ?? 0:00.01 [sh] 0 702 1 0 23 0 39572 3128016 select Ds ?? 0:00.00 [ruby20] 0 775 1 0 20 0 14788 2553632 kqread Ds ?? 0:00.00 [master] 0 803 1 0 52 0 13476 274368 select Ds ?? 0:00.00 [sshd] 0 816 1 0 20 0 9596 4002368 nanslp Ds ?? 0:00.00 [cron] 0 850 1 0 20 0 10104 293584 wait Ds ?? 0:00.00 [login] 0 851 1 0 52 0 9552 140464 ttyin Ds+ ?? 0:00.00 [getty] 0 852 1 0 52 0 9552 3126128 ttyin Ds+ ?? 0:00.00 [getty] 0 853 1 0 52 0 9552 3122352 ttyin Ds+ ?? 0:00.00 [getty] 0 854 1 0 52 0 9552 2554576 ttyin Ds+ ?? 0:00.00 [getty] 0 855 1 0 52 0 9552 4010864 ttyin Ds+ ?? 0:00.00 [getty] 0 856 1 0 52 0 9552 3115744 ttyin Ds+ ?? 0:00.00 [getty] 0 857 1 0 52 0 9552 396000 ttyin Ds+ ?? 0:00.00 [getty] 0 1526 624 0 52 0 11444 3118576 ttyout D+ ?? 0:00.00 [pkg_add] 0 1712 850 0 40 0 9880 144240 pause D ?? 0:00.00 [csh] 0 2029 1712 0 21 0 9860 142352 wait D+ ?? 0:00.00 [sh] 0 2191 2029 0 24 0 22036 205344 zio->io_ D+ ?? 0:00.00 [svn] ------------------------------------------------------------------------ vmstat -s 2062835 cpu context switches 218945 device interrupts 102246 software interrupts 253558 traps 6446774 system calls 17 kernel threads created 1751 fork() calls 425 vfork() calls 0 rfork() calls 0 swap pager pageins 0 swap pager pages paged in 0 swap pager pageouts 0 swap pager pages paged out 820 vnode pager pageins 5195 vnode pager pages paged in 0 vnode pager pageouts 0 vnode pager pages paged out 0 page daemon wakeups 0 pages examined by the page daemon 0 pages reactivated 60326 copy-on-write faults 123 copy-on-write optimized faults 152566 zero fill pages zeroed 6660 zero fill pages prezeroed 1 intransit blocking page faults 241112 total VM faults taken 0 pages affected by kernel thread creation 1681354 pages affected by fork() 406094 pages affected by vfork() 0 pages affected by rfork() 0 pages cached 1269061 pages freed 0 pages freed by daemon 0 pages freed by exiting processes 6729 pages active 65239 pages inactive 0 pages in VM cache 114047 pages wired down 693767 pages free 4096 bytes per page 3756175 total name lookups cache hits (92% pos + 2% neg) system 0% per-directory deletions 1%, falsehits 0%, toolong 0% ------------------------------------------------------------------------ vmstat -m Type InUse MemUse HighUse Requests Size(s) cdev 8 1K - 8 128 sigio 1 1K - 1 32 filedesc 48 13K - 2300 16,256,512,1024 kdtrace 318 72K - 3051 64,256 kenv 84 7K - 94 16,32,64,128,4096 kqueue 4 3K - 748 128,1024 proc-args 26 2K - 1865 16,32,64,128,256 hhook 2 1K - 2 128 ithread 67 6K - 67 16,64,128 CAM periph 6 1K - 23 16,32,64,128 KTRACE 100 13K - 100 128 linker 181 1835K - 206 16,32,256,1024,4096 lockf 20 2K - 275586 32,64 loginclass 2 1K - 16 64 ip6ndp 3 1K - 3 64,128 temp 17 225K - 5457 16,32,64,128,256,512,1024,2048,4096 devbuf 1294 2959K - 1353 16,32,64,128,256,512,1024,2048,4096 module 532 34K - 532 64,128 mtx_pool 2 8K - 2 4096 osd 3 1K - 11 16,32 kbdmux 6 18K - 6 16,256,1024,2048 pmchooks 1 1K - 1 64 entropy 1024 64K - 1024 64 LED 2 1K - 2 16,64 subproc 135 195K - 2286 256,4096 proc 2 8K - 2 4096 session 21 2K - 40 64 pgrp 23 2K - 44 64 cred 136 13K - 207018 64,128 uidinfo 4 2K - 15 64,1024 plimit 12 3K - 328 256 acpiintr 1 1K - 1 32 acpica 2082 120K - 36589 16,32,64,128,256,512,1024,4096 sysctltmp 0 0K - 290 16,32,64,128,4096 sysctloid 2611 80K - 2755 16,32,64,128 sysctl 0 0K - 352 16,32,64 tidhash 1 8K - 1 callout 3 768K - 3 umtx 670 63K - 670 64,128 p1003.1b 1 1K - 1 16 SWAP 4 281K - 4 64 bus-sc 41 58K - 1485 16,64,128,256,512,1024,2048,4096 bus 1074 46K - 3263 16,32,64,128,256,1024 devstat 4 9K - 4 16,4096 eventhandler 98 5K - 98 32,64,128 kobj 376 752K - 560 2048 Per-cpu 1 1K - 1 16 acpitask 1 4K - 1 4096 md_disk 1 2K - 1 2048 rman 106 7K - 502 32,64 sbuf 0 0K - 922 16,32,64,128,256,512,1024,2048,4096 DEVFS1 74 19K - 79 256 stack 0 0K - 2 128 taskqueue 83 4K - 295 16,32,64 Unitno 15 1K - 1023 16,64 iov 0 0K - 1360 16,64,128,256,512,1024 select 50 4K - 50 64 ioctlops 0 0K - 2544 16,32,64,128,256,512,1024 msg 4 25K - 4 1024,4096 sem 4 101K - 4 1024,4096 shm 1 12K - 1 tty 18 9K - 20 512,1024 pts 1 1K - 1 128 mbuf_tag 0 0K - 17 32 shmfd 1 4K - 1 4096 DEVFS3 91 12K - 103 128,256 pcb 21 79K - 516 16,64,512,1024,2048,4096 soname 28 4K - 1657 16,32,128 acl 0 0K - 579 4096 vfscache 1 1024K - 1 cl_savebuf 0 0K - 231 32 vfs_hash 1 512K - 1 vnodes 4 1K - 4 32,128 mount 116 3K - 546 16,32,64,128,256 vnodemarker 0 0K - 1436 512 BPF 7 9K - 7 64,128,256,4096 ether_multi 17 1K - 24 16,32,64 ifaddr 33 8K - 34 16,32,64,128,256,512,2048 ifnet 3 3K - 3 64,1024 DEVFS 15 1K - 16 16,64 clone 6 24K - 6 4096 arpcom 1 1K - 1 16 lltable 9 2K - 10 128,256 DEVFSP 1 1K - 1 32 routetbl 33 4K - 173 16,32,64,128,256 igmp 2 1K - 2 128 in_multi 2 1K - 3 128 sctp_iter 0 0K - 2 256 sctp_ifn 2 1K - 2 128 sctp_ifa 4 1K - 4 128 sctp_vrf 1 1K - 1 64 sctp_a_it 0 0K - 2 16 hostcache 1 16K - 1 syncache 1 72K - 1 in6_multi 15 2K - 15 16,256 mld 2 1K - 2 128 rpc 2 1K - 2 256 audit_evclass 180 3K - 219 16 jblocks 2 1K - 2 128 savedino 0 0K - 1483 256 sbdep 0 0K - 59 32 jsegdep 8 1K - 55328 32 jseg 8 1K - 1061 128 jfreefrag 0 0K - 1360 64 jnewblk 0 0K - 13664 64 jmvref 0 0K - 3 64 jremref 0 0K - 31718 64 jaddref 0 0K - 8586 64 freedep 0 0K - 84 32 freework 1 1K - 29575 128 newdirblk 0 0K - 659 32 dirrem 0 0K - 29872 64 mkdir 0 0K - 1314 64 diradd 0 0K - 7272 64 freefile 1 1K - 27577 32 freeblks 0 0K - 26893 128 freefrag 0 0K - 1360 64 indirdep 0 0K - 59 64 newblk 1 1024K - 13665 128 bmsafemap 1 4K - 5145 128,4096 inodedep 46 524K - 33641 256 pagedep 1 64K - 3630 128 ufs_dirhash 36 18K - 135 16,32,64,128,512 ufs_quota 1 512K - 1 ufs_mount 3 13K - 3 256,4096 vm_pgdata 3 65K - 3 64 UMAHash 1 1K - 1 256 atkbddev 2 1K - 2 32 pfs_nodes 21 3K - 21 128 GEOM 69 9K - 631 16,32,64,128,256,512,1024,2048 feeder 7 1K - 7 16 CAM dev queue 6 1K - 6 32 acpisem 16 2K - 16 64,128 CAM path 10 1K - 44 16 CAM CCB 45 90K - 65 2048 raid_data 0 0K - 84 16,128 CAM DEV 8 8K - 19 1024 CAM XPT 28 2K - 110 16,32,64,256,1024 md_nvidia_data 0 0K - 14 512 md_sii_data 0 0K - 14 512 CAM queue 17 3K - 68 16,256 acpidev 27 1K - 27 32 apmdev 1 1K - 1 64 madt_table 0 0K - 1 2048 CAM SIM 6 1K - 6 128 pci_link 8 1K - 8 16,128 io_apic 1 1K - 1 1024 MCA 4 1K - 4 128 nexusdev 4 1K - 4 16 isadev 11 1K - 11 64 solaris 195384 328382K - 13974311 16,32,64,128,256,512,1024,2048,4096 kstat_data 6 1K - 6 32 ------------------------------------------------------------------------ vmstat -z ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP UMA Kegs: 128, 0, 100, 20, 100, 0, 0 UMA Zones: 224, 0, 100, 2, 100, 0, 0 UMA Slabs: 284, 0, 31397, 33, 139472, 0, 0 UMA RCntSlabs: 544, 0, 682, 4, 722, 0, 0 UMA Hash: 128, 0, 3, 27, 4, 0, 0 16 Bucket: 76, 0, 61, 89, 111, 0, 0 32 Bucket: 140, 0, 66, 74, 137, 7, 0 64 Bucket: 268, 0, 55, 113, 247, 17, 0 128 Bucket: 524, 0, 244, 141, 34087,1252, 0 VM OBJECT: 148, 0, 36257, 221, 106060, 0, 0 MAP: 144, 0, 8, 46, 8, 0, 0 KMAP ENTRY: 76, 157150, 64, 836, 312198, 0, 0 MAP ENTRY: 76, 0, 655, 545, 97880, 0, 0 fakepg: 72, 0, 0, 0, 0, 0, 0 mt_zone: 2060, 0, 333, 40, 333, 0, 0 16: 16, 0, 36891, 461, 638742, 0, 0 32: 32, 0, 30310, 19297, 6329882, 0, 0 64: 64, 0, 36737, 492, 1522910, 0, 0 128: 128, 0, 30340, 530, 533489, 0, 0 256: 256, 0, 731, 754, 185411, 0, 0 512: 512, 0, 29811, 341, 5308607, 0, 0 1024: 1024, 0, 9666, 718, 60745, 0, 0 2048: 2048, 0, 16396, 198, 28963, 0, 0 4096: 4096, 0, 89, 269, 98086, 0, 0 Files: 56, 0, 163, 440, 155479, 0, 0 rl_entry: 28, 0, 70, 565, 70, 0, 0 TURNSTILE: 72, 0, 336, 84, 336, 0, 0 umtx pi: 52, 0, 0, 0, 0, 0, 0 MAC labels: 20, 0, 0, 0, 0, 0, 0 PROC: 728, 0, 42, 48, 2193, 0, 0 THREAD: 752, 0, 274, 61, 856, 0, 0 SLEEPQUEUE: 44, 0, 336, 195, 336, 0, 0 VMSPACE: 236, 0, 26, 166, 2178, 0, 0 cpuset: 40, 0, 201, 443, 779, 0, 0 audit_record: 824, 0, 0, 0, 0, 0, 0 mbuf_packet: 256, 168765, 260, 1156, 92456, 0, 0 mbuf: 256, 168765, 43, 733, 64891, 0, 0 mbuf_cluster: 2048, 26368, 1024, 340, 1792, 0, 0 mbuf_jumbo_page: 4096, 13184, 0, 0, 0, 0, 0 mbuf_jumbo_9k: 9216, 11718, 0, 0, 0, 0, 0 mbuf_jumbo_16k: 16384, 8788, 0, 0, 0, 0, 0 mbuf_ext_refcnt: 4, 0, 0, 0, 0, 0, 0 ttyinq: 152, 0, 135, 47, 270, 0, 0 ttyoutq: 256, 0, 72, 48, 144, 0, 0 g_bio: 152, 0, 0, 650, 398987, 0, 0 ata_request: 208, 0, 0, 0, 0, 0, 0 ata_composite: 180, 0, 0, 0, 0, 0, 0 vtnet_tx_hdr: 16, 0, 0, 0, 0, 0, 0 VNODE: 284, 0, 36288, 308, 68787, 0, 0 VNODEPOLL: 60, 0, 0, 0, 0, 0, 0 S VFS Cache: 72, 0, 35167, 290, 110902, 0, 0 STS VFS Cache: 92, 0, 0, 0, 0, 0, 0 L VFS Cache: 292, 0, 574, 50, 3135, 0, 0 LTS VFS Cache: 312, 0, 0, 0, 0, 0, 0 NAMEI: 1024, 0, 1, 91, 626136, 0, 0 DIRHASH: 1024, 0, 0, 116, 205, 0, 0 NCLNODE: 376, 0, 0, 0, 0, 0, 0 Mountpoints: 676, 0, 12, 36, 22, 0, 0 pipe: 400, 0, 30, 60, 603, 0, 0 ksiginfo: 80, 0, 84, 204, 151, 0, 0 itimer: 220, 0, 1, 35, 1, 0, 0 KNOTE: 72, 0, 50, 321, 5809, 0, 0 socket: 416, 111987, 55, 71, 819, 0, 0 ipq: 32, 904, 0, 0, 0, 0, 0 udp_inpcb: 252, 111990, 8, 112, 472, 0, 0 udpcb: 8, 112056, 8, 601, 472, 0, 0 tcp_inpcb: 252, 111990, 7, 128, 184, 0, 0 tcpcb: 688, 111985, 7, 58, 184, 0, 0 tcptw: 52, 22464, 0, 144, 51, 0, 0 syncache: 120, 15360, 0, 0, 0, 0, 0 hostcache: 76, 15400, 2, 98, 2, 0, 0 tcpreass: 20, 1690, 0, 0, 0, 0, 0 sackhole: 20, 0, 0, 0, 0, 0, 0 sctp_ep: 940, 111984, 0, 0, 0, 0, 0 sctp_asoc: 1508, 40000, 0, 0, 0, 0, 0 sctp_laddr: 24, 80040, 0, 290, 3, 0, 0 sctp_raddr: 496, 80000, 0, 0, 0, 0, 0 sctp_chunk: 96, 400000, 0, 0, 0, 0, 0 sctp_readq: 76, 400000, 0, 0, 0, 0, 0 sctp_stream_msg_out: 68, 400008, 0, 0, 0, 0, 0 sctp_asconf: 24, 400055, 0, 0, 0, 0, 0 sctp_asconf_ack: 24, 400055, 0, 0, 0, 0, 0 ripcb: 252, 111990, 1, 29, 1, 0, 0 unpcb: 172, 111987, 37, 101, 152, 0, 0 rtentry: 108, 0, 13, 167, 14, 0, 0 selfd: 28, 0, 59, 576, 16208, 0, 0 SWAPMETA: 276, 131082, 0, 0, 0, 0, 0 FFS inode: 116, 0, 6849, 6087, 34428, 0, 0 FFS1 dinode: 128, 0, 0, 0, 0, 0, 0 FFS2 dinode: 256, 0, 6849, 5691, 34428, 0, 0 taskq_zone: 24, 0, 0, 725, 993, 0, 0 range_seg_cache: 48, 0, 6656, 1534, 133965, 0, 0 zio_cache: 732, 0, 252, 518, 1073121, 0, 0 zio_link_cache: 24, 0, 252, 618, 620076, 0, 0 lz4_ctx: 16384, 0, 0, 0, 0, 0, 0 sa_cache: 44, 0, 29380, 272, 34268, 0, 0 dnode_t: 480, 0, 35071, 57, 36940, 0, 0 dmu_buf_impl_t: 136, 0, 45873, 527, 145701, 0, 0 arc_buf_hdr_t: 148, 0, 17307, 14257, 120603, 0, 0 arc_buf_t: 40, 0, 16176, 752, 138309, 0, 0 zil_lwb_cache: 168, 0, 0, 0, 0, 0, 0 zfs_znode_cache: 252, 0, 29380, 125, 34268, 0, 0 ------------------------------------------------------------------------ vmstat -i interrupt total rate irq1: atkbd0 348 1 irq17: em0 53188 247 irq23: ahci0 165409 769 cpu0:timer 94596 439 cpu2:timer 33798 157 cpu1:timer 33016 153 cpu3:timer 32532 151 Total 412887 1920 ------------------------------------------------------------------------ pstat -T 163/111981 files 0M/2047M swap space ------------------------------------------------------------------------ pstat -s Device 512-blocks Used Avail Capacity /dev/ada0p3 2096696 0 2096696 0% /dev/md0 2096896 0 2096896 0% Total 4193592 0 4193592 0% ------------------------------------------------------------------------ iostat iostat: kvm_read(_tk_nin): invalid address (0x0) iostat: disabling TTY statistics md0 ada0 ada1 cpu KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s us ni sy in id 0.00 0 0.00 19.01 215 3.99 31.52 601 18.51 0 0 2 0 97 ------------------------------------------------------------------------ ipcs -a Message Queues: T ID KEY MODE OWNER GROUP CREATOR CGROUP CBYTES QNUM QBYTES LSPID LRPID STIME RTIME CTIME Shared Memory: T ID KEY MODE OWNER GROUP CREATOR CGROUP NATTCH SEGSZ CPID LPID ATIME DTIME CTIME Semaphores: T ID KEY MODE OWNER GROUP CREATOR CGROUP NSEMS OTIME CTIME ------------------------------------------------------------------------ ipcs -T msginfo: msgmax: 16384 (max characters in a message) msgmni: 40 (# of message queues) msgmnb: 2048 (max characters in a message queue) msgtql: 40 (max # of messages in system) msgssz: 8 (size of a message segment) msgseg: 2048 (# of message segments in system) shminfo: shmmax: 536870912 (max shared memory segment size) shmmin: 1 (min shared memory segment size) shmmni: 192 (max number of shared memory identifiers) shmseg: 128 (max shared memory segments per process) shmall: 131072 (max amount of shared memory in pages) seminfo: semmni: 50 (# of semaphore identifiers) semmns: 340 (# of semaphores in system) semmnu: 150 (# of undo structures in system) semmsl: 340 (max # of semaphores per id) semopm: 100 (max # of operations per semop call) semume: 50 (max # of undo entries per process) semusz: 616 (size in bytes of undo structure) semvmx: 32767 (semaphore maximum value) semaem: 16384 (adjust on exit max value) ------------------------------------------------------------------------ nfsstat Client Info: Rpc Counts: Getattr Setattr Lookup Readlink Read Write Create Remove 0 0 0 0 0 0 0 0 Rename Link Symlink Mkdir Rmdir Readdir RdirPlus Access 0 0 0 0 0 0 0 0 Mknod Fsstat Fsinfo PathConf Commit 0 0 0 0 0 Rpc Info: TimedOut Invalid X Replies Retries Requests 0 0 0 0 0 Cache Info: Attr Hits Misses Lkup Hits Misses BioR Hits Misses BioW Hits Misses 0 0 0 0 0 0 0 0 BioRLHits Misses BioD Hits Misses DirE Hits Misses Accs Hits Misses 0 0 0 0 0 0 0 0 Server Info: Getattr Setattr Lookup Readlink Read Write Create Remove 0 0 0 0 0 0 0 0 Rename Link Symlink Mkdir Rmdir Readdir RdirPlus Access 0 0 0 0 0 0 0 0 Mknod Fsstat Fsinfo PathConf Commit 0 0 0 0 0 Server Ret-Failed 0 Server Faults 0 Server Cache Stats: Inprog Idem Non-idem Misses 0 0 0 0 Server Write Gathering: WriteOps WriteRPC Opsaved 0 0 0 ------------------------------------------------------------------------ netstat -s tcp: 62688 packets sent 422 data packets (211437 bytes) 1 data packet (550 bytes) retransmitted 0 data packets unnecessarily retransmitted 0 resends initiated by MTU discovery 41781 ack-only packets (327 delayed) 0 URG only packets 0 window probe packets 20224 window update packets 260 control packets 87664 packets received 678 acks (for 211566 bytes) 121 duplicate acks 0 acks for unsent data 87361 packets (91096102 bytes) received in-sequence 2 completely duplicate packets (551 bytes) 0 old duplicate packets 0 packets with some dup. data (0 bytes duped) 0 out-of-order packets (0 bytes) 0 packets (0 bytes) of data after window 0 window probes 3 window update packets 0 packets received after close 0 discarded for bad checksums 0 discarded for bad header offset fields 0 discarded because packet too short 0 discarded due to memory problems 131 connection requests 0 connection accepts 0 bad connection attempts 0 listen queue overflows 0 ignored RSTs in the windows 131 connections established (including accepts) 177 connections closed (including 0 drops) 59 connections updated cached RTT on close 59 connections updated cached RTT variance on close 29 connections updated cached ssthresh on close 0 embryonic connections dropped 678 segments updated rtt (of 671 attempts) 1 retransmit timeout 0 connections dropped by rexmit timeout 0 persist timeouts 0 connections dropped by persist timeout 0 Connections (fin_wait_2) dropped because of timeout 0 keepalive timeouts 0 keepalive probes sent 0 connections dropped by keepalive 11 correct ACK header predictions 86849 correct data packet header predictions 0 syncache entries added 0 retransmitted 0 dupsyn 0 dropped 0 completed 0 bucket overflow 0 cache overflow 0 reset 0 stale 0 aborted 0 badack 0 unreach 0 zone failures 0 cookies sent 0 cookies received 2 hostcache entries added 0 bucket overflow 0 SACK recovery episodes 0 segment rexmits in SACK recovery episodes 0 byte rexmits in SACK recovery episodes 0 SACK options (SACK blocks) received 0 SACK options (SACK blocks) sent 0 SACK scoreboard overflow 0 packets with ECN CE bit set 0 packets with ECN ECT(0) bit set 0 packets with ECN ECT(1) bit set 0 successful ECN handshakes 0 times ECN reduced the congestion window udp: 638 datagrams received 0 with incomplete header 0 with bad data length field 0 with bad checksum 0 with no checksum 0 dropped due to no socket 223 broadcast/multicast datagrams undelivered 0 dropped due to full socket buffers 0 not for hashed pcb 415 delivered 441 datagrams output 0 times multicast source filter matched ip: 88358 total packets received 0 bad header checksums 0 with size smaller than minimum 0 with data size < data length 0 with ip length > max ip packet size 0 with header length < data size 0 with data length < header length 0 with bad options 0 with incorrect version number 0 fragments received 0 fragments dropped (dup or out of space) 0 fragments dropped after timeout 0 packets reassembled ok 88312 packets for this host 27 packets for unknown/unsupported protocol 0 packets forwarded (0 packets fast forwarded) 19 packets not forwardable 0 packets received for unknown multicast group 0 redirects sent 63139 packets sent from this host 0 packets sent with fabricated ip header 0 output packets dropped due to no bufs, etc. 0 output packets discarded due to no route 0 output datagrams fragmented 0 fragments created 0 datagrams that can't be fragmented 0 tunneling packets that can't find gif 0 datagrams with bad address in header icmp: 0 calls to icmp_error 0 errors not generated in response to an icmp message Output histogram: echo reply: 10 0 messages with bad code fields 0 messages less than the minimum length 0 messages with bad checksum 0 messages with bad length 0 multicast echo requests ignored 0 multicast timestamp requests ignored Input histogram: destination unreachable: 12 echo: 10 10 message responses generated 0 invalid return addresses 0 no return routes igmp: 15 messages received 0 messages received with too few bytes 0 messages received with wrong TTL 0 messages received with bad checksum 15 V1/V2 membership queries received 0 V3 membership queries received 0 membership queries received with invalid field(s) 15 general queries received 0 group queries received 0 group-source queries received 0 group-source queries dropped 0 membership reports received 0 membership reports received with invalid field(s) 0 membership reports received for groups to which we belong 0 V3 reports received without Router Alert 0 membership reports sent arp: 4 ARP requests sent 10 ARP replies sent 3446 ARP requests received 3 ARP replies received 3472 ARP packets received 0 total packets dropped due to no ARP entry 1 ARP entry timed out 0 Duplicate IPs seen ip6: 0 total packets received 0 with size smaller than minimum 0 with data size < data length 0 with bad options 0 with incorrect version number 0 fragments received 0 fragments dropped (dup or out of space) 0 fragments dropped after timeout 0 fragments that exceeded limit 0 packets reassembled ok 0 packets for this host 0 packets forwarded 0 packets not forwardable 0 redirects sent 0 packets sent from this host 0 packets sent with fabricated ip header 0 output packets dropped due to no bufs, etc. 4 output packets discarded due to no route 0 output datagrams fragmented 0 fragments created 0 datagrams that can't be fragmented 0 packets that violated scope rules 0 multicast packets which we don't join Mbuf statistics: 0 one mbuf 0 one ext mbuf 0 two or more ext mbuf 0 packets whose headers are not contiguous 0 tunneling packets that can't find gif 0 packets discarded because of too many headers 0 failures of source address selection Source addresses selection rule applied: icmp6: 0 calls to icmp6_error 0 errors not generated in response to an icmp6 message 0 errors not generated because of rate limitation 0 messages with bad code fields 0 messages < minimum length 0 bad checksums 0 messages with bad length Histogram of error messages to be generated: 0 no route 0 administratively prohibited 0 beyond scope 0 address unreachable 0 port unreachable 0 packet too big 0 time exceed transit 0 time exceed reassembly 0 erroneous header field 0 unrecognized next header 0 unrecognized option 0 redirect 0 unknown 0 message responses generated 0 messages with too many ND options 0 messages with bad ND options 0 bad neighbor solicitation messages 0 bad neighbor advertisement messages 0 bad router solicitation messages 0 bad router advertisement messages 0 bad redirect messages 0 path MTU changes rip6: 0 messages received 0 checksum calculations on inbound 0 messages with bad checksum 0 messages dropped due to no socket 0 multicast messages dropped due to no socket 0 messages dropped due to full socket buffers 0 delivered 0 datagrams output ------------------------------------------------------------------------ netstat -m 303/1889/2192 mbufs in use (current/cache/total) 18446744073709551484/1496/1364/26368 mbuf clusters in use (current/cache/total/max) 260/1156 mbuf+clusters out of packet secondary zone in use (current/cache) 0/0/0/13184 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/11718 9k jumbo clusters in use (current/cache/total/max) 0/0/0/8788 16k jumbo clusters in use (current/cache/total/max) 18014398509481795K/3464K/3276K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile 0 calls to protocol drain routines ------------------------------------------------------------------------ netstat -idW Name Mtu Network Address Ipkts Ierrs Idrop Opkts Oerrs Coll Drop em0 1500 08:00:27:dd:96:ff 91830 0 0 63154 0 0 0 em0 1500 10.10.0.0 10.10.0.226 88294 - - 63139 - - - lo0 16384 0 0 0 0 0 0 0 lo0 16384 localhost ::1 0 - - 0 - - - lo0 16384 fe80::1%lo0 fe80::1 0 - - 0 - - - lo0 16384 your-net localhost 0 - - 0 - - - ------------------------------------------------------------------------ netstat -anr Routing tables Internet: Destination Gateway Flags Refs Use Netif Expire default 10.10.0.252 UGS 0 3461 em0 10.10.0.0/24 link#1 U 0 59678 em0 10.10.0.226 link#1 UHS 0 0 lo0 127.0.0.1 link#2 UH 0 0 lo0 Internet6: Destination Gateway Flags Netif Expire ::/96 ::1 UGRS lo0 ::1 link#2 UH lo0 ::ffff:0.0.0.0/96 ::1 UGRS lo0 fe80::/10 ::1 UGRS lo0 fe80::%lo0/64 link#2 U lo0 fe80::1%lo0 link#2 UHS lo0 ff01::%lo0/32 ::1 U lo0 ff02::/16 ::1 UGRS lo0 ff02::%lo0/32 ::1 U lo0 ------------------------------------------------------------------------ netstat -anA Active Internet connections (including servers) Tcpcb Proto Recv-Q Send-Q Local Address Foreign Address (state) ca15eac0 tcp4 0 0 10.10.0.226.47238 113.52.8.158.80 CLOSE_WAIT ccd30810 tcp4 4771 0 10.10.0.226.41077 10.10.0.201.80 CLOSE_WAIT ccd2fac0 tcp4 0 0 *.24 *.* LISTEN ccd30000 tcp6 0 0 *.24 *.* LISTEN ca15e560 tcp4 0 0 *.25 *.* LISTEN ca06cac0 tcp4 0 0 *.873 *.* LISTEN ca15e000 tcp6 0 0 *.873 *.* LISTEN c9c03dc8 udp4 0 0 127.0.0.1.123 *.* c9c03ccc udp6 0 0 fe80:2::1.123 *.* c9c035e8 udp6 0 0 ::1.123 *.* c9c037e0 udp4 0 0 10.10.0.226.123 *.* c9c038dc udp6 0 0 *.123 *.* c9c039d8 udp4 0 0 *.123 *.* c9c03ad4 udp4 0 0 *.514 *.* c9c03bd0 udp6 0 0 *.514 *.* Active UNIX domain sockets Address Type Recv-Q Send-Q Inode Conn Refs Nextref Addr c9d15ac0 stream 2 0 0 0 0 0 public/qmgr c9d10204 stream 2 0 0 0 0 0 public/pickup cd65e810 stream 0 0 cd64d470 0 0 0 private/scache cd65e8bc stream 0 0 cd64d58c 0 0 0 private/anvil cd65e968 stream 0 0 cd64d6a8 0 0 0 private/lmtp c9d12158 stream 0 0 cd64d7c4 0 0 0 private/virtual c9d120ac stream 0 0 cd64d8e0 0 0 0 private/local c9d12000 stream 0 0 cd64d9fc 0 0 0 private/discard c9d11ec8 stream 0 0 cd64db18 0 0 0 private/retry c9d1060c stream 0 0 cd64dc34 0 0 0 private/error c9d110ac stream 0 0 cd64dd50 0 0 0 public/showq c9d11158 stream 0 0 cd64de6c 0 0 0 private/relay c9d11204 stream 0 0 cd64f000 0 0 0 private/smtp c9d112b0 stream 0 0 cd64f11c 0 0 0 private/proxywrite c9d1135c stream 0 0 cd64f238 0 0 0 private/proxymap c9d11408 stream 0 0 cd64f354 0 0 0 public/flush c9d114b4 stream 0 0 c9e55470 0 0 0 private/verify c9d11560 stream 0 0 cd539238 0 0 0 private/trace c9d1160c stream 0 0 cd539354 0 0 0 private/defer c9d116b8 stream 0 0 cd539470 0 0 0 private/bounce c9d11764 stream 0 0 cd53958c 0 0 0 private/rewrite c9d11810 stream 0 0 cd5396a8 0 0 0 private/tlsmgr c9d11a14 stream 0 0 cd5397c4 0 0 0 public/qmgr c9d11ac0 stream 0 0 cd5398e0 0 0 0 public/cleanup c9d11cc4 stream 0 0 cd5399fc 0 0 0 public/pickup c9de62b0 stream 0 0 0 c9de635c 0 0 c9de635c stream 7796 0 0 c9de62b0 0 0 c9de6408 stream 0 0 0 c9de64b4 0 0 c9de64b4 stream 0 0 0 c9de6408 0 0 c9d10ec8 stream 0 0 c9bca7c4 0 0 0 /var/run/devd.pipe c9d15d70 dgram 0 0 0 c9d11d70 0 c9d12968 c9d12968 dgram 0 0 0 c9d11d70 0 c9de6204 c9de6204 dgram 0 0 0 c9d11d70 0 c9d10764 c9d10764 dgram 0 0 0 c9d11d70 0 c9d10810 c9d10810 dgram 0 0 0 c9d11d70 0 0 c9d11d70 dgram 0 0 c8505b18 0 c9d15d70 0 /var/run/logpriv c9d11e1c dgram 0 0 c8505c34 0 0 0 /var/run/log ------------------------------------------------------------------------ netstat -aL Current listen queue sizes (qlen/incqlen/maxqlen) Proto Listen Local Address tcp4 0/0/128 *.24 tcp6 0/0/128 *.24 tcp4 0/0/100 *.smtp tcp4 0/0/5 *.rsync tcp6 0/0/5 *.rsync unix 0/0/100 private/scache unix 0/0/100 private/anvil unix 0/0/100 private/lmtp unix 0/0/100 private/virtual unix 0/0/100 private/local unix 0/0/100 private/discard unix 0/0/100 private/retry unix 0/0/100 private/error unix 0/0/100 public/showq unix 0/0/100 private/relay unix 0/0/100 private/smtp unix 0/0/100 private/proxywrite unix 0/0/100 private/proxymap unix 0/0/100 public/flush unix 0/0/100 private/verify unix 0/0/100 private/trace unix 0/0/100 private/defer unix 0/0/100 private/bounce unix 0/0/100 private/rewrite unix 0/0/100 private/tlsmgr unix 1/0/100 public/qmgr unix 0/0/100 public/cleanup unix 1/0/100 public/pickup unix 0/0/4 /var/run/devd.pipe ------------------------------------------------------------------------ fstat fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 USER CMD PID FD MOUNT INUM MODE SZ|DV R/W root svn 2191 root / 2 drwxr-xr-x 1024 r root svn 2191 wd - - error - root svn 2191 text / 562141 -rwxr-xr-x 237152 r root svn 2191 ctty /dev 39 crw------- ttyv0 rw root svn 2191 0 /dev 39 crw------- ttyv0 rw root svn 2191 1 /dev 39 crw------- ttyv0 rw root svn 2191 2 /dev 39 crw------- ttyv0 rw root svn 2191 3 - - error - root svn 2191 5* internet stream tcp ca15eac0 root svn 2191 6 / 562052 -rw------- 0 rw root sh 2029 root / 2 drwxr-xr-x 1024 r root sh 2029 wd - - error - root sh 2029 text / 963203 -r-xr-xr-x 121840 r root sh 2029 ctty /dev 39 crw------- ttyv0 rw root sh 2029 0 /dev 39 crw------- ttyv0 rw root sh 2029 1 /dev 39 crw------- ttyv0 rw root sh 2029 2 /dev 39 crw------- ttyv0 rw root csh 1712 root / 2 drwxr-xr-x 1024 r root csh 1712 wd - - error - root csh 1712 text / 963186 -r-xr-xr-x 340912 r root csh 1712 ctty /dev 39 crw------- ttyv0 rw root csh 1712 15 /dev 39 crw------- ttyv0 rw root csh 1712 16 /dev 39 crw------- ttyv0 rw root csh 1712 17 /dev 39 crw------- ttyv0 rw root csh 1712 18 /dev 39 crw------- ttyv0 rw root csh 1712 19 /dev 39 crw------- ttyv0 rw root pkg_add 1526 root / 2 drwxr-xr-x 1024 r root pkg_add 1526 wd / 402154 drwx------ 512 r root pkg_add 1526 text / 1050958 -r-xr-xr-x 62932 r root pkg_add 1526 ctty /dev 77 crw--w---- pts/0 rw root pkg_add 1526 0 /dev 77 crw--w---- pts/0 rw root pkg_add 1526 1 /dev 77 crw--w---- pts/0 rw root pkg_add 1526 2 /dev 77 crw--w---- pts/0 rw root pkg_add 1526 3* internet stream tcp ccd30810 root pkg_add 1526 4 / 561796 prwx------ 0 rw root pkg_add 1526 5 / 561800 prwx------ 0 rw root getty 857 root / 2 drwxr-xr-x 1024 r root getty 857 wd / 2 drwxr-xr-x 1024 r root getty 857 text / 1053381 -r-xr-xr-x 22948 r root getty 857 ctty /dev 46 crw------- ttyv7 rw root getty 857 0 /dev 46 crw------- ttyv7 rw root getty 857 1 /dev 46 crw------- ttyv7 rw root getty 857 2 /dev 46 crw------- ttyv7 rw root getty 856 root / 2 drwxr-xr-x 1024 r root getty 856 wd / 2 drwxr-xr-x 1024 r root getty 856 text / 1053381 -r-xr-xr-x 22948 r root getty 856 ctty /dev 45 crw------- ttyv6 rw root getty 856 0 /dev 45 crw------- ttyv6 rw root getty 856 1 /dev 45 crw------- ttyv6 rw root getty 856 2 /dev 45 crw------- ttyv6 rw root getty 855 root / 2 drwxr-xr-x 1024 r root getty 855 wd / 2 drwxr-xr-x 1024 r root getty 855 text / 1053381 -r-xr-xr-x 22948 r root getty 855 ctty /dev 44 crw------- ttyv5 rw root getty 855 0 /dev 44 crw------- ttyv5 rw root getty 855 1 /dev 44 crw------- ttyv5 rw root getty 855 2 /dev 44 crw------- ttyv5 rw root getty 854 root / 2 drwxr-xr-x 1024 r root getty 854 wd / 2 drwxr-xr-x 1024 r root getty 854 text / 1053381 -r-xr-xr-x 22948 r root getty 854 ctty /dev 43 crw------- ttyv4 rw root getty 854 0 /dev 43 crw------- ttyv4 rw root getty 854 1 /dev 43 crw------- ttyv4 rw root getty 854 2 /dev 43 crw------- ttyv4 rw root getty 853 root / 2 drwxr-xr-x 1024 r root getty 853 wd / 2 drwxr-xr-x 1024 r root getty 853 text / 1053381 -r-xr-xr-x 22948 r root getty 853 ctty /dev 42 crw------- ttyv3 rw root getty 853 0 /dev 42 crw------- ttyv3 rw root getty 853 1 /dev 42 crw------- ttyv3 rw root getty 853 2 /dev 42 crw------- ttyv3 rw root getty 852 root / 2 drwxr-xr-x 1024 r root getty 852 wd / 2 drwxr-xr-x 1024 r root getty 852 text / 1053381 -r-xr-xr-x 22948 r root getty 852 ctty /dev 41 crw------- ttyv2 rw root getty 852 0 /dev 41 crw------- ttyv2 rw root getty 852 1 /dev 41 crw------- ttyv2 rw root getty 852 2 /dev 41 crw------- ttyv2 rw root getty 851 root / 2 drwxr-xr-x 1024 r root getty 851 wd / 2 drwxr-xr-x 1024 r root getty 851 text / 1053381 -r-xr-xr-x 22948 r root getty 851 ctty /dev 40 crw------- ttyv1 rw root getty 851 0 /dev 40 crw------- ttyv1 rw root getty 851 1 /dev 40 crw------- ttyv1 rw root getty 851 2 /dev 40 crw------- ttyv1 rw root login 850 root / 2 drwxr-xr-x 1024 r root login 850 wd / 160512 drwxr-xr-x 512 r root login 850 text / 1053945 -r-sr-xr-x 23496 r root login 850 ctty /dev 39 crw------- ttyv0 rw root login 850 0 /dev 39 crw------- ttyv0 rw root login 850 1 /dev 39 crw------- ttyv0 rw root login 850 2 /dev 39 crw------- ttyv0 rw root login 850 3* local dgram c9d15d70 <-> c9d11d70 root cron 816 root / 2 drwxr-xr-x 1024 r root cron 816 wd / 240789 drwxr-x--- 512 r root cron 816 text / 1050804 -r-xr-xr-x 35092 r root cron 816 0 /dev 20 crw-rw-rw- null rw root cron 816 1 /dev 20 crw-rw-rw- null rw root cron 816 2 /dev 20 crw-rw-rw- null rw root cron 816 3 / 241332 -rw------- 3 w root sshd 803 root / 2 drwxr-xr-x 1024 r root sshd 803 wd / 2 drwxr-xr-x 1024 r root sshd 803 text / 1050139 -r-xr-xr-x 268892 r root sshd 803 0 /dev 20 crw-rw-rw- null rw root sshd 803 1 /dev 20 crw-rw-rw- null rw root sshd 803 2 /dev 20 crw-rw-rw- null rw root sshd 803 3* internet6 stream tcp ccd30000 root sshd 803 4* internet stream tcp ccd2fac0 root master 775 root / 2 drwxr-xr-x 1024 r root master 775 wd / 643027 drwxr-xr-x 512 r root master 775 text / 1299666 -rwxr-xr-x 125828 r root master 775 0 /dev 20 crw-rw-rw- null rw root master 775 1 /dev 20 crw-rw-rw- null rw root master 775 2 /dev 20 crw-rw-rw- null rw root master 775 3* local dgram c9d12968 <-> c9d11d70 root master 775 4 /dev 20 crw-rw-rw- null rw root master 775 5* pipe cd661e10 <-> cd661ecc 0 rw root master 775 6 /dev 20 crw-rw-rw- null rw root master 775 7 /dev 20 crw-rw-rw- null rw root master 775 8 /dev 20 crw-rw-rw- null rw root master 775 9 /dev 20 crw-rw-rw- null rw root master 775 10 / 649029 -rw------- 17 rw root master 775 11 / 649030 -rw------- 17 rw root master 775 12* pipe cd650000 <-> cd6500bc 0 rw root master 775 13* internet stream tcp ca15e560 root master 775 15* pipe c84cf320 <-> c84cf3dc 0 rw root master 775 16* pipe c84cf3dc <-> c84cf320 0 rw root master 775 17* local stream c9d11cc4 root master 775 18* pipe c84cf640 <-> c84cf6fc 0 rw root master 775 19* pipe c84cf6fc <-> c84cf640 0 rw root master 775 21* local stream c9d11ac0 root master 775 22* pipe c84d0320 <-> c84d03dc 0 rw root master 775 23* pipe c84d03dc <-> c84d0320 0 rw root master 775 24* local stream c9d11a14 root master 775 25* pipe c84cf190 <-> c84cf24c 0 rw root master 775 26* pipe c84cf24c <-> c84cf190 0 rw root master 775 28* local stream c9d11810 root master 775 29* pipe c84d0960 <-> c84d0a1c 0 rw root master 775 30* pipe c84d0a1c <-> c84d0960 0 rw root master 775 31* local stream c9d11764 root master 775 32* pipe c9995960 <-> c9995a1c 0 rw root master 775 33* pipe c9995a1c <-> c9995960 0 rw root master 775 34* local stream c9d116b8 root master 775 35* pipe c84cf4b0 <-> c84cf56c 0 rw root master 775 36* pipe c84cf56c <-> c84cf4b0 0 rw root master 775 37* local stream c9d1160c root master 775 38* pipe c9995000 <-> c99950bc 0 rw root master 775 39* pipe c99950bc <-> c9995000 0 rw root master 775 40* local stream c9d11560 root master 775 41* pipe c8506af0 <-> c8506bac 0 rw root master 775 42* pipe c8506bac <-> c8506af0 0 rw root master 775 43* local stream c9d114b4 root master 775 44* pipe c9995320 <-> c99953dc 0 rw root master 775 45* pipe c99953dc <-> c9995320 0 rw root master 775 46* local stream c9d11408 root master 775 47* pipe cd651640 <-> cd6516fc 0 rw root master 775 48* pipe cd6516fc <-> cd651640 0 rw root master 775 49* local stream c9d1135c root master 775 50* pipe cd6514b0 <-> cd65156c 0 rw root master 775 51* pipe cd65156c <-> cd6514b0 0 rw root master 775 52* local stream c9d112b0 root master 775 53* pipe cd651320 <-> cd6513dc 0 rw root master 775 54* pipe cd6513dc <-> cd651320 0 rw root master 775 55* local stream c9d11204 root master 775 56* pipe cd651190 <-> cd65124c 0 rw root master 775 57* pipe cd65124c <-> cd651190 0 rw root master 775 58* local stream c9d11158 root master 775 59* pipe cd651000 <-> cd6510bc 0 rw root master 775 60* pipe cd6510bc <-> cd651000 0 rw root master 775 61* local stream c9d110ac root master 775 62* pipe cd650e10 <-> cd650ecc 0 rw root master 775 63* pipe cd650ecc <-> cd650e10 0 rw root master 775 64* local stream c9d1060c root master 775 65* pipe cd650c80 <-> cd650d3c 0 rw root master 775 66* pipe cd650d3c <-> cd650c80 0 rw root master 775 67* local stream c9d11ec8 root master 775 68* pipe cd650af0 <-> cd650bac 0 rw root master 775 69* pipe cd650bac <-> cd650af0 0 rw root master 775 70* local stream c9d12000 root master 775 71* pipe cd650960 <-> cd650a1c 0 rw root master 775 72* pipe cd650a1c <-> cd650960 0 rw root master 775 73* local stream c9d120ac root master 775 74* pipe cd6507d0 <-> cd65088c 0 rw root master 775 75* pipe cd65088c <-> cd6507d0 0 rw root master 775 76* local stream c9d12158 root master 775 77* pipe cd650640 <-> cd6506fc 0 rw root master 775 78* pipe cd6506fc <-> cd650640 0 rw root master 775 79* local stream cd65e968 root master 775 80* pipe cd6504b0 <-> cd65056c 0 rw root master 775 81* pipe cd65056c <-> cd6504b0 0 rw root master 775 82* local stream cd65e8bc root master 775 83* pipe cd650320 <-> cd6503dc 0 rw root master 775 84* pipe cd6503dc <-> cd650320 0 rw root master 775 85* local stream cd65e810 root master 775 86* pipe cd650190 <-> cd65024c 0 rw root master 775 87* pipe cd65024c <-> cd650190 0 rw root master 775 88* pipe cd6500bc <-> cd650000 0 rw root master 775 89* pipe cd662000 <-> cd6620bc 100 rw root master 775 90* pipe cd6620bc <-> cd662000 0 rw root master 775 91* pipe cd661ecc <-> cd661e10 0 rw root ruby20 702 root / 2 drwxr-xr-x 1024 r root ruby20 702 wd / 2 drwxr-xr-x 1024 r root ruby20 702 text / 644795 -rwxr-xr-x 4220 r root ruby20 702 0 /dev 20 crw-rw-rw- null r root ruby20 702 1 /dev 20 crw-rw-rw- null w root ruby20 702 2 /dev 20 crw-rw-rw- null w root ruby20 702 3* pipe c9995190 <-> c999524c 0 rw root ruby20 702 4* pipe c999524c <-> c9995190 0 rw root ruby20 702 5* pipe c99957d0 <-> c999588c 0 rw root ruby20 702 6* pipe c999588c <-> c99957d0 0 rw root ruby20 702 7* local dgram c9de6204 <-> c9d11d70 root sh 624 root / 2 drwxr-xr-x 1024 r root sh 624 wd / 561792 drwxrwxrwt 512 r root sh 624 text / 963203 -r-xr-xr-x 121840 r root sh 624 ctty /dev 77 crw--w---- pts/0 rw root sh 624 0 /dev 77 crw--w---- pts/0 rw root sh 624 1 /dev 77 crw--w---- pts/0 rw root sh 624 2 /dev 77 crw--w---- pts/0 rw root sh 624 4 / 561796 prwx------ 0 rw root sh 624 5 / 561800 prwx------ 0 rw root sh 624 10 / 561893 -rw------- 11904 r root rsync 593 root / 2 drwxr-xr-x 1024 r root rsync 593 wd / 2 drwxr-xr-x 1024 r root rsync 593 text / 1530928 -rwxr-xr-x 372708 r root rsync 593 0 /dev 20 crw-rw-rw- null rw root rsync 593 1 /dev 20 crw-rw-rw- null rw root rsync 593 2 /dev 20 crw-rw-rw- null rw root rsync 593 4* internet6 stream tcp ca15e000 root rsync 593 5* internet stream tcp ca06cac0 root ntpd 579 root / 2 drwxr-xr-x 1024 r root ntpd 579 wd / 2 drwxr-xr-x 1024 r root ntpd 579 text / 1050854 -r-xr-xr-x 368260 r root ntpd 579 0 /dev 20 crw-rw-rw- null rw root ntpd 579 1 /dev 20 crw-rw-rw- null rw root ntpd 579 2 /dev 20 crw-rw-rw- null rw root ntpd 579 3* local dgram c9d10764 <-> c9d11d70 root ntpd 579 5 / 242870 -rw-r--r-- 71702 w root ntpd 579 20* internet dgram udp c9c039d8 root ntpd 579 21* internet6 dgram udp c9c038dc root ntpd 579 22* internet dgram udp c9c037e0 root ntpd 579 23* internet6 dgram udp c9c035e8 root ntpd 579 24* internet6 dgram udp c9c03ccc root ntpd 579 25* internet dgram udp c9c03dc8 root ntpd 579 26* route raw 0 c9c09000 root sh 538 root / 2 drwxr-xr-x 1024 r root sh 538 wd / 561792 drwxrwxrwt 512 r root sh 538 text / 963203 -r-xr-xr-x 121840 r root sh 538 ctty /dev 77 crw--w---- pts/0 rw root sh 538 0 /dev 77 crw--w---- pts/0 rw root sh 538 1 /dev 77 crw--w---- pts/0 rw root sh 538 2 /dev 77 crw--w---- pts/0 rw root sh 538 4 / 561796 prwx------ 0 rw root sh 538 5 / 561800 prwx------ 0 rw root sh 538 10 / 561853 -rwx--x--x 11904 r root sempty 537 root / 2 drwxr-xr-x 1024 r root sempty 537 wd / 561792 drwxrwxrwt 512 r root sempty 537 text / 6022 -r-xr-xr-x 16716 r root sempty 537 0 - - bad - root sempty 537 1 - - bad - root sempty 537 2 - - bad - root sempty 537 3* local dgram c9d10810 <-> c9d11d70 root sempty 537 4 / 561796 prwx------ 0 rw root sempty 537 5 / 561800 prwx------ 0 rw root sempty 537 6* pseudo-terminal master pts/0 rw root sempty 537 7 /dev 77 crw--w---- pts/0 rw root syslogd 452 root / 2 drwxr-xr-x 1024 r root syslogd 452 wd / 2 drwxr-xr-x 1024 r root syslogd 452 text / 1051059 -r-xr-xr-x 36628 r root syslogd 452 0 /dev 20 crw-rw-rw- null rw root syslogd 452 1 /dev 20 crw-rw-rw- null rw root syslogd 452 2 /dev 20 crw-rw-rw- null rw root syslogd 452 3 / 242220 -rw------- 3 w root syslogd 452 4* local dgram c9d11e1c root syslogd 452 5* local dgram c9d11d70 root syslogd 452 6* internet6 dgram udp c9c03bd0 root syslogd 452 7* internet dgram udp c9c03ad4 root syslogd 452 8 /dev 9 crw------- klog r root syslogd 452 10 - - bad - root syslogd 452 11 / 240796 -rw-r--r-- 62862 w root syslogd 452 12 / 252530 -rw------- 61 w root syslogd 452 13 / 241600 -rw------- 62391 w root syslogd 452 14 / 241993 -rw-r----- 39282 w root syslogd 452 15 / 252526 -rw-r--r-- 61 w root syslogd 452 16 / 252531 -rw------- 61 w root syslogd 452 17 / 241866 -rw------- 78386 w root syslogd 452 18 / 252525 -rw------- 2683 w root syslogd 452 19 / 252529 -rw-r----- 61 w root devd 363 root / 2 drwxr-xr-x 1024 r root devd 363 wd / 2 drwxr-xr-x 1024 r root devd 363 text / 963162 -r-xr-xr-x 419976 r root devd 363 0 /dev 20 crw-rw-rw- null rw root devd 363 1 /dev 20 crw-rw-rw- null rw root devd 363 2 /dev 20 crw-rw-rw- null rw root devd 363 3 /dev 5 crw------- devctl r root devd 363 4* local stream c9d10ec8 root devd 363 5 / 242091 -rw------- 3 w _dhcp dhclient 350 root / 240778 dr-xr-xr-x 512 r _dhcp dhclient 350 wd / 240778 dr-xr-xr-x 512 r _dhcp dhclient 350 jail / 240778 dr-xr-xr-x 512 r _dhcp dhclient 350 text / 963142 -r-xr-xr-x 78956 r _dhcp dhclient 350 0 /dev 20 crw-rw-rw- null rw _dhcp dhclient 350 1 /dev 20 crw-rw-rw- null rw _dhcp dhclient 350 2 /dev 20 crw-rw-rw- null rw _dhcp dhclient 350 3 / 241956 -rw------- 3 w _dhcp dhclient 350 5* route raw 0 c9bf8b60 _dhcp dhclient 350 6* pipe c85060bc <-> c8506000 0 rw _dhcp dhclient 350 7 / 252517 ---------- 1356 w _dhcp dhclient 350 8 /dev 11 crw------- bpf rw _dhcp dhclient 350 9* internet raw ip c9d1a000 root dhclient 312 root / 2 drwxr-xr-x 1024 r root dhclient 312 wd / 2 drwxr-xr-x 1024 r root dhclient 312 text / 963142 -r-xr-xr-x 78956 r root dhclient 312 0 /dev 20 crw-rw-rw- null rw root dhclient 312 1 /dev 20 crw-rw-rw- null rw root dhclient 312 2 /dev 20 crw-rw-rw- null rw root dhclient 312 3 / 241956 -rw------- 3 w root dhclient 312 5* pipe c8506000 <-> c85060bc 0 rw root md0 131 root / 2 drwxr-xr-x 1024 r root md0 131 wd / 2 drwxr-xr-x 1024 r root zfskern 33 root / 2 drwxr-xr-x 1024 r root zfskern 33 wd / 2 drwxr-xr-x 1024 r root init 1 root / 2 drwxr-xr-x 1024 r root init 1 wd / 2 drwxr-xr-x 1024 r root init 1 text / 966430 -r-xr-xr-x 734904 r root kernel 0 root / 2 drwxr-xr-x 1024 r root kernel 0 wd / 2 drwxr-xr-x 1024 r ------------------------------------------------------------------------ dmesg Copyright (c) 1992-2014 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 9.3-RELEASE-p10 #0: Tue Feb 24 21:01:19 UTC 2015 root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC i386 gcc version 4.2.1 20070831 patched [FreeBSD] CPU: Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz (3299.33-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x306a9 Family = 0x6 Model = 0x3a Stepping = 9 Features=0x1783fbff Features2=0x201 AMD Features=0x28100800 AMD Features2=0x1 real memory = 3690921984 (3519 MB) avail memory = 3598864384 (3432 MB) Event timer "LAPIC" quality 400 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs FreeBSD/SMP: 1 package(s) x 4 core(s) cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 cpu2 (AP): APIC ID: 2 cpu3 (AP): APIC ID: 3 pnpbios: Bad PnP BIOS data checksum ioapic0 irqs 0-23 on motherboard kbd1 at kbdmux0 acpi0: on motherboard acpi0: Power Button (fixed) acpi0: Sleep Button (fixed) Timecounter "HPET" frequency 14318180 Hz quality 950 cpu0: on acpi0 cpu1: on acpi0 cpu2: on acpi0 cpu3: on acpi0 attimer0: port 0x40-0x43,0x50-0x53 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 atrtc0: port 0x70-0x71 on acpi0 Event timer "RTC" frequency 32768 Hz quality 0 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <32-bit timer at 3.579545MHz> port 0x4008-0x400b on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 vgapci0: mem 0xe0000000-0xe07fffff irq 18 at device 2.0 on pci0 vgapci0: Boot video device pci0: at device 7.0 (no driver attached) em0: port 0xd020-0xd027 mem 0xf0420000-0xf043ffff irq 17 at device 17.0 on pci0 em0: Ethernet address: 08:00:27:dd:96:ff pcib1: at device 24.0 on pci0 pci1: on pcib1 pcib2: at device 25.0 on pci0 pci2: on pcib2 isab0: at device 31.0 on pci0 isa0: on isab0 ahci0: port 0xe000-0xe007,0xe008-0xe00b,0xe010-0xe017,0xe018-0xe01b,0xe020-0xe02f mem 0xf0500000-0xf0501fff irq 23 at device 31.2 on pci0 ahci0: AHCI v1.10 with 3 3Gbps ports, Port Multiplier not supported ahcich0: at channel 0 on ahci0 ahcich1: at channel 1 on ahci0 ahcich2: at channel 2 on ahci0 acpi_acad0: on acpi0 atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] psm0: irq 12 on atkbdc0 psm0: [GIANT-LOCKED] psm0: model IntelliMouse Explorer, device ID 4 pmtimer0 on isa0 orm0: at iomem 0xc0000-0xc7fff pnpid ORM0000 on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 ata0: at port 0x1f0-0x1f7,0x3f6 irq 14 on isa0 ata1: at port 0x170-0x177,0x376 irq 15 on isa0 ppc0: parallel port not found. smist0: on cpu0 device_attach: smist0 attach returned 6 smist1: on cpu1 device_attach: smist1 attach returned 6 smist2: on cpu2 device_attach: smist2 attach returned 6 smist3: on cpu3 device_attach: smist3 attach returned 6 Timecounters tick every 10.000 msec ada0 at ahcich0 bus 0 scbus0 target 0 lun 0 ada0: ATA-6 SATA 2.x device ada0: Serial Number VB02dfdfa5-15d661ec ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada0: Command Queueing enabled ada0: 51200MB (104857600 512 byte sectors: 16H 63S/T 16383C) ada0: Previously was known as ad4 ada1 at ahcich1 bus 0 scbus1 target 0 lun 0 ada1: ATA-6 SATA 2.x device ada1: Serial Number VB58541d87-cebb8b3c ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada1: Command Queueing enabled ada1: 51200MB (104857600 512 byte sectors: 16H 63S/T 16383C) ada1: Previously was known as ad6 SMP: AP CPU #2 Launched! SMP: AP CPU #1 Launched! SMP: AP CPU #3 Launched! Trying to mount root from ufs:/dev/ada0p2 [rw]... Setting hostuuid: 70a2dc7d-2ef6-48bc-a4c6-202115bb958c. Setting hostid: 0x93c93c34. ZFS NOTICE: Prefetch is disabled by default on i386 -- to enable, add "vfs.zfs.prefetch_disable=0" to /boot/loader.conf. ZFS WARNING: Recommended minimum kmem_size is 512MB; expect unstable behavior. Consider tuning vm.kmem_size and vm.kmem_size_max in /boot/loader.conf. ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Entropy harvesting: interrupts ethernet point_to_point kickstart. Starting file system checks: /dev/ada0p2: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ada0p2: clean, 10464716 free (33284 frags, 1303929 blocks, 0.3% fragmentation) Mounting local file systems:. Adding /swap as additional swap Setting hostname: 93i386. Starting Network: lo0 em0. lo0: flags=8049 metric 0 mtu 16384 options=600003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 inet 127.0.0.1 netmask 0xff000000 nd6 options=21 em0: flags=8843 metric 0 mtu 1500 options=9b ether 08:00:27:dd:96:ff nd6 options=29 media: Ethernet autoselect (1000baseT ) status: active Starting devd. Starting dhclient. DHCPREQUEST on em0 to 255.255.255.255 port 67 DHCPACK from 10.10.0.252 unknown dhcp option value 0x80 bound to 10.10.0.226 -- renewal in 43200 seconds. add net fe80::: gateway ::1 add net ff02::: gateway ::1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib a.out ldconfig path: /usr/lib/aout /usr/lib/compat/aout Creating and/or trimming log files. Starting syslogd. No core dumps found. Clearing /tmp. Starting local daemons:sysctl: unknown oid 'vfs.zfs.vdev.min_pending' sysctl: unknown oid 'vfs.zfs.vdev.max_pending' Updating '.': At revision 3367. Updating '.': Fetching http://colossus.isux.com/pub/FreeBSD/ports/i386/packages-9.3-release/Latest/pkg.tbz... Done. pkg_add: package 'pkg-1.4.12' or its older version already installed Error: Unable to get http://colossus.isux.com/pub/FreeBSD/ports/i386/packages-9.3-release/Latest/pkgng.tbz: Not Found pkg_add: unable to fetch 'http://colossus.isux.com/pub/FreeBSD/ports/i386/packages-9.3-release/Latest/pkgng.tbz' by URL Starting update... . Updating motd:. Starting ntpd. Starting rsyncd. Apr 1 17:51:58 93i386 postfix/postfix-script[601]: fatal: the Postfix mail system is not running Apr 1 17:52:03 93i386 postfix/postfix-script[647]: fatal: the Postfix mail system is not running \^[[1;31mWarning: Setting modulepath is deprecated in puppet.conf. See http://links.puppetlabs.com/env-settings-deprecations (at /usr/local/lib/ruby/site_ruby/2.0/puppet/settings.rb:1095:in `block in issue_deprecations')\^[[0m \^[[1;31mWarning: Setting manifestdir is deprecated. See http://links.puppetlabs.com/env-settings-deprecations (at /usr/local/lib/ruby/site_ruby/2.0/puppet/settings.rb:1095:in `block in issue_deprecations')\^[[0m \^[[1;31mWarning: Setting manifest is deprecated in puppet.conf. See http://links.puppetlabs.com/env-settings-deprecations (at /usr/local/lib/ruby/site_ruby/2.0/puppet/settings.rb:1095:in `block in issue_deprecations')\^[[0m \^[[1;31mWarning: Setting templatedir is deprecated. See http://links.puppetlabs.com/env-settings-deprecations (at /usr/local/lib/ruby/site_ruby/2.0/puppet/settings.rb:1095:in `block in issue_deprecations')\^[[0m Starting puppet. postfix/postfix-script: starting the Postfix mail system Configuring syscons: blanktime. Performing sanity check on sshd configuration. Starting sshd. Starting cron. postfix/postfix-script: fatal: the Postfix mail system is already running Apr 1 17:53:10 93i386 postfix/postfix-script[815]: fatal: the Postfix mail system is already running Starting background file system checks in 60 seconds. Wed Apr 1 17:53:13 CEST 2015 Apr 1 17:54:04 93i386 postfix/pickup[776]: fatal: scan_dir_push: open directory maildrop: No such file or directory Apr 1 17:55:04 93i386 master[1528]: fatal: master_spawn: exec /usr/local/libexec/postfix/pickup: No such file or directory Apr 1 17:56:05 93i386 master[1710]: fatal: master_spawn: exec /usr/local/libexec/postfix/pickup: No such file or directory Apr 1 17:57:06 93i386 master[1711]: fatal: master_spawn: exec /usr/local/libexec/postfix/pickup: No such file or directory Apr 1 17:57:27 93i386 login: ROOT LOGIN (root) ON ttyv0 Apr 1 17:58:04 93i386 postfix/qmgr[777]: fatal: scan_dir_push: open directory incoming: No such file or directory Apr 1 17:58:07 93i386 master[1797]: fatal: master_spawn: exec /usr/local/libexec/postfix/pickup: No such file or directory Apr 1 17:59:08 93i386 master[2051]: fatal: master_spawn: exec /usr/local/libexec/postfix/pickup: No such file or directory Apr 1 18:00:09 93i386 master[2071]: fatal: master_spawn: exec /usr/local/libexec/postfix/pickup: No such file or directory Apr 1 18:01:10 93i386 master[2077]: fatal: master_spawn: exec /usr/local/libexec/postfix/pickup: No such file or directory Apr 1 18:02:11 93i386 master[2080]: fatal: master_spawn: exec /usr/local/libexec/postfix/pickup: No such file or directory Apr 1 18:03:04 93i386 master[2083]: fatal: master_spawn: exec /usr/local/libexec/postfix/qmgr: No such file or directory Apr 1 18:03:12 93i386 master[2084]: fatal: master_spawn: exec /usr/local/libexec/postfix/pickup: No such file or directory Apr 1 18:04:05 93i386 master[2085]: fatal: master_spawn: exec /usr/local/libexec/postfix/qmgr: No such file or directory Apr 1 18:04:13 93i386 master[2086]: fatal: master_spawn: exec /usr/local/libexec/postfix/pickup: No such file or directory Apr 1 18:05:06 93i386 master[2089]: fatal: master_spawn: exec /usr/local/libexec/postfix/qmgr: No such file or directory Apr 1 18:05:14 93i386 master[2090]: fatal: master_spawn: exec /usr/local/libexec/postfix/pickup: No such file or directory Apr 1 18:06:07 93i386 master[2091]: fatal: master_spawn: exec /usr/local/libexec/postfix/qmgr: No such file or directory Apr 1 18:06:15 93i386 master[2092]: fatal: master_spawn: exec /usr/local/libexec/postfix/pickup: No such file or directory Apr 1 18:07:08 93i386 master[2096]: fatal: master_spawn: exec /usr/local/libexec/postfix/qmgr: No such file or directory Apr 1 18:07:16 93i386 master[2097]: fatal: master_spawn: exec /usr/local/libexec/postfix/pickup: No such file or directory Apr 1 18:08:09 93i386 master[2103]: fatal: master_spawn: exec /usr/local/libexec/postfix/qmgr: No such file or directory Apr 1 18:08:18 93i386 master[2104]: fatal: master_spawn: exec /usr/local/libexec/postfix/pickup: No such file or directory Apr 1 18:09:10 93i386 master[2106]: fatal: master_spawn: exec /usr/local/libexec/postfix/qmgr: No such file or directory Apr 1 18:09:19 93i386 master[2108]: fatal: master_spawn: exec /usr/local/libexec/postfix/pickup: No such file or directory Apr 1 18:10:11 93i386 master[2113]: fatal: master_spawn: exec /usr/local/libexec/postfix/qmgr: No such file or directory Apr 1 18:10:20 93i386 master[2114]: fatal: master_spawn: exec /usr/local/libexec/postfix/pickup: No such file or directory Apr 1 18:11:12 93i386 master[2130]: fatal: master_spawn: exec /usr/local/libexec/postfix/qmgr: No such file or directory Apr 1 18:11:21 93i386 master[2132]: fatal: master_spawn: exec /usr/local/libexec/postfix/pickup: No such file or directory Apr 1 18:12:13 93i386 master[2133]: fatal: master_spawn: exec /usr/local/libexec/postfix/qmgr: No such file or directory Apr 1 18:12:22 93i386 master[2134]: fatal: master_spawn: exec /usr/local/libexec/postfix/pickup: No such file or directory Apr 1 18:13:14 93i386 master[2137]: fatal: master_spawn: exec /usr/local/libexec/postfix/qmgr: No such file or directory Apr 1 18:13:23 93i386 master[2138]: fatal: master_spawn: exec /usr/local/libexec/postfix/pickup: No such file or directory Apr 1 18:14:15 93i386 master[2140]: fatal: master_spawn: exec /usr/local/libexec/postfix/qmgr: No such file or directory Apr 1 18:14:24 93i386 master[2141]: fatal: master_spawn: exec /usr/local/libexec/postfix/pickup: No such file or directory Apr 1 18:15:16 93i386 master[2144]: fatal: master_spawn: exec /usr/local/libexec/postfix/qmgr: No such file or directory Apr 1 18:15:25 93i386 master[2146]: fatal: master_spawn: exec /usr/local/libexec/postfix/pickup: No such file or directory Apr 1 18:16:17 93i386 master[2148]: fatal: master_spawn: exec /usr/local/libexec/postfix/qmgr: No such file or directory Apr 1 18:16:26 93i386 master[2149]: fatal: master_spawn: exec /usr/local/libexec/postfix/pickup: No such file or directory Apr 1 18:17:18 93i386 master[2151]: fatal: master_spawn: exec /usr/local/libexec/postfix/qmgr: No such file or directory Apr 1 18:17:27 93i386 master[2152]: fatal: master_spawn: exec /usr/local/libexec/postfix/pickup: No such file or directory Apr 1 18:18:19 93i386 master[2154]: fatal: master_spawn: exec /usr/local/libexec/postfix/qmgr: No such file or directory Apr 1 18:18:28 93i386 master[2155]: fatal: master_spawn: exec /usr/local/libexec/postfix/pickup: No such file or directory Apr 1 18:19:20 93i386 master[2158]: fatal: master_spawn: exec /usr/local/libexec/postfix/qmgr: No such file or directory Apr 1 18:19:29 93i386 master[2159]: fatal: master_spawn: exec /usr/local/libexec/postfix/pickup: No such file or directory Apr 1 18:20:21 93i386 master[2172]: fatal: master_spawn: exec /usr/local/libexec/postfix/qmgr: No such file or directory Apr 1 18:20:30 93i386 master[2173]: fatal: master_spawn: exec /usr/local/libexec/postfix/pickup: No such file or directory Apr 1 18:21:22 93i386 master[2177]: fatal: master_spawn: exec /usr/local/libexec/postfix/qmgr: No such file or directory Apr 1 18:21:31 93i386 master[2178]: fatal: master_spawn: exec /usr/local/libexec/postfix/pickup: No such file or directory Apr 1 18:22:23 93i386 master[2192]: fatal: master_spawn: exec /usr/local/libexec/postfix/qmgr: No such file or directory Apr 1 18:22:32 93i386 master[2193]: fatal: master_spawn: exec /usr/local/libexec/postfix/pickup: No such file or directory panic: kmem_malloc(86016): kmem_map too small: 431890432 total allocated cpuid = 2 KDB: stack backtrace: #0 0xc0b0f96f at kdb_backtrace+0x4f #1 0xc0ad65af at panic+0x16f #2 0xc0d62c7a at kmem_malloc+0x28a #3 0xc0d56387 at page_alloc+0x27 #4 0xc0d58a30 at uma_large_malloc+0x50 #5 0xc0abdf3c at malloc+0x8c #6 0xc86caf50 at zfs_kmem_alloc+0x20 #7 0xc860dd34 at zio_buf_alloc+0x44 #8 0xc85ef8cd at vdev_queue_io_to_issue+0x95d #9 0xc85efbbc at vdev_queue_io_done+0xec #10 0xc860d511 at zio_vdev_io_done+0x81 #11 0xc860da4e at zio_execute+0xee #12 0xc0b1c6f2 at taskqueue_run_locked+0xd2 #13 0xc0b1d4dc at taskqueue_thread_loop+0xbc #14 0xc0aa243f at fork_exit+0xcf #15 0xc0f88734 at fork_trampoline+0x8 Uptime: 32m19s Physical memory: 3499 MB Dumping 537 MB: 522 506 490 474 458 442 426 410 394 378 362 346 330 314 298 282 266 250 234 218 202 186 170 154 138 122 106 90 74 58 42 26 10 ------------------------------------------------------------------------ kernel config options CONFIG_AUTOGENERATED ident GENERIC machine i386 cpu I686_CPU cpu I586_CPU cpu I486_CPU makeoptions WITH_CTF=1 makeoptions DEBUG=-gdwarf-2 options USB_DEBUG options RDRAND_RNG options PADLOCK_RNG options AH_SUPPORT_AR5416 options IEEE80211_SUPPORT_MESH options IEEE80211_AMPDU_AGE options IEEE80211_DEBUG options SC_PIXEL_MODE options VESA options AHD_REG_PRETTY_PRINT options AHC_REG_PRETTY_PRINT options ATA_STATIC_ID options ATA_CAM options SMP options DDB_CTF options KDB_TRACE options KDB options INCLUDE_CONFIG_FILE options KDTRACE_HOOKS options MAC options AUDIT options HWPMC_HOOKS options KBD_INSTALL_CDEV options PRINTF_BUFR_SIZE=128 options _KPOSIX_PRIORITY_SCHEDULING options SYSVSEM options SYSVMSG options SYSVSHM options STACK options KTRACE options SCSI_DELAY=5000 options COMPAT_FREEBSD7 options COMPAT_FREEBSD6 options COMPAT_FREEBSD5 options COMPAT_FREEBSD4 options GEOM_LABEL options GEOM_RAID options GEOM_PART_GPT options PSEUDOFS options PROCFS options CD9660 options MSDOSFS options NFS_ROOT options NFSLOCKD options NFSD options NFSCL options MD_ROOT options QUOTA options UFS_GJOURNAL options UFS_DIRHASH options UFS_ACL options SOFTUPDATES options FFS options SCTP options TCP_OFFLOAD options INET6 options INET options PREEMPTION options SCHED_ULE options NEW_PCIB options NATIVE options GEOM_PART_MBR options GEOM_PART_EBR_COMPAT options GEOM_PART_EBR options GEOM_PART_BSD options ISAPNP device isa device npx device mem device io device uart_ns8250 device atpic device apic device cpufreq device acpi device eisa device pci device fdc device ahci device ata device mvs device siis device ahb device ahc device ahd device esp device hptiop device isp device mpt device mps device mpr device sym device trm device adv device adw device aha device aic device bt device ncv device nsp device stg device isci device scbus device ch device da device sa device cd device pass device ses device amr device arcmsr device asr device ciss device dpt device hptmv device hptnr device hptrr device hpt27xx device iir device ips device mly device twa device tws device aac device aacp device aacraid device ida device mfi device mlx device pst device twe device atkbdc device atkbd device psm device kbdmux device vga device splash device sc device agp device pmtimer device cbb device pccard device cardbus device uart device ppc device ppbus device lpt device plip device ppi device puc device bxe device de device em device igb device ixgb device le device ti device txp device vx device miibus device ae device age device alc device ale device bce device bfe device bge device cas device dc device et device fxp device gem device hme device jme device lge device msk device nfe device nge device pcn device re device rl device sf device sge device sis device sk device ste device stge device tl device tx device vge device vr device vte device wb device xl device cs device ed device ex device ep device fe device ie device sn device xe device wlan device wlan_wep device wlan_ccmp device wlan_tkip device wlan_amrr device an device ath device ath_pci device ath_hal device ath_rate_sample device ipw device iwi device iwn device malo device mwl device ral device wi device wpi device loop device random device ether device vlan device tun device pty device md device gif device faith device firmware device bpf device uhci device ohci device ehci device xhci device usb device uhid device ukbd device ulpt device umass device ums device urio device u3g device uark device ubsa device uftdi device uipaq device uplcom device uslcom device uvisor device uvscom device aue device axe device cdce device cue device kue device rue device udav device rum device run device uath device upgt device ural device urtw device zyd device sound device snd_cmi device snd_csa device snd_emu10kx device snd_es137x device snd_hda device snd_ich device snd_uaudio device snd_via8233 device virtio device virtio_pci device vtnet device virtio_blk device virtio_scsi device virtio_balloon ------------------------------------------------------------------------ ddb capture buffer ddb: ddb_capture: kvm_nlist