osquery An Operating System Instrumentation Framewor

1. Getting Started
2. install guide for OS X and Linux
3. Features Overview
4. Logging
5. query example

1. running processes
2. loaded kernel modules
3. open network connections
4. browser plugins
5. hardware events or file hashes 

http://osquery.readthedocs.org/en/stable/
http://www.oschina.net/p/osquery
http://www.infoq.com/cn/news/2014/10/osquery-facebook-sql

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys C9D8B80B
sudo add-apt-repository "deb [arch=amd64] https://osquery-packages.s3.amazonaws.com/trusty trusty main"
sudo apt-get update
sudo apt-get install osquery

sudo rpm -ivh https://osquery-packages.s3.amazonaws.com/centos6/noarch/osquery-s3-centos6-repo-1-0.0.noarch.rpm
sudo yum install osquery

http://osquery.readthedocs.org/en/stable/installation/install-linux/

1. osqueryd
The high-performance and low-footprint distributed host monitoring daemon, osqueryd, allows you to schedule queries to be executed across your entire infrastructure. The daemon takes care of aggregating the query results over time and generates logs which indicate state changes in your infrastructure. You can use this to maintain insight into the security, performance, configuration, and state of your entire infrastructure. osqueryd‘s logging can integrate into your internal log aggregation pipeline, regardless of your technology stack, via a robust plugin architecture.

2. osqueryi
The interactive query console, gives you a SQL interface to try out new queries and explore your operating system. With the power of a complete SQL language and dozens of useful tables built-in, osqueryi is an invaluable tool when performing incident response, diagnosing an systems operations problem, troubleshooting a performance issue, etc.

3. osquery 
osquery is cross platform. Even though osquery takes advantage of very low-level operating system APIs, you can build and use osquery on Mac OS X, Ubuntu, Cent OS and other popular enterprise Linux distributions

4. plugin architecture
To assist with the rollout process, the osquery user guide has detailed documentation on internal deployment. osquery was built so that every environment specific aspect of the toolchain can be hot-swapped at run-time with custom plugins. Use these interfaces to deeply integrate osquery into your infrastructure if one of the several existing plugins don not suit your needs

1. 是采用了定时采样的方式收集主机上相关信息，适合在大量的分布式集群上部署agent，在一个中心server上部署analysis进行数据集中分析
2. osquery使用了用户态的系统API获取系统上相关信息，包括
    1) running processes(运行中进程)
    2) loaded kernel modules
    3) open network connections(网络外连)
    4) browser plugins
    5) hardware events or file hashes 
3. 对于入侵检测有帮助的字段有"running processes"、"open network connections"，但是osquery的采样方式觉得它获取的不是实时的准确数据，因为恶意的"running processes"和"open network connections"都有可能是瞬间发生的
4. osquery有一个beta版的内核态采样模块，但是处于unstable状态，工作较不稳定，而且工作方式依然是定时采样，而不是实时的hook

https://osquery.readthedocs.org/en/stable/introduction/overview/

1. Status logs (info, warning, error, and fatal)
2. Query schedule results logs

https://osquery.readthedocs.org/en/stable/deployment/logging/

1. List the the users:
SELECT * FROM users;

2. Check the processes that have a deleted executable:
SELECT * FROM processes WHERE on_disk = 0;

3. Get the process name, port, and PID, which are listening on all interfaces:
SELECT DISTINCT process.name, listening.port, process.pid
FROM processes AS process
JOIN listening_ports AS listening ON process.pid = listening.pid
WHERE listening.address = ‘0.0.0.0‘;

4. Find every OS X LaunchDaemon that launches an executable and keeps it running:
SELECT name, program || program_arguments AS executable
FROM launchd
WHERE
  (run_at_load = ‘true‘ AND keep_alive = ‘true‘)
AND
  (program != ‘‘ OR program_arguments != ‘‘);

5. Check for ARP anomalies from the host‘s perspective:
SELECT address, mac, count(mac) AS mac_count
FROM arp_cache GROUP BY mac
HAVING count(mac) > 1;

6. Alternatively, you could also use a SQL sub-query to accomplish the same result:
SELECT address, mac, mac_count
FROM
  (SELECT address, mac, count(mac) AS mac_count FROM arp_cache GROUP BY mac)
WHERE mac_count > 1;

https://github.com/facebook/osquery