René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Centralised logging via rsyslog","articleBody":"##### ISSUE TYPE\r\n * Feature Idea\r\n\r\n##### COMPONENT NAME\r\n~~~\r\nSystemvm\r\n~~~\r\n\r\n##### CLOUDSTACK VERSION\r\n~~~\r\nIt can be merge to 4.14\r\n~~~\r\n\r\n##### CONFIGURATION\r\n\u003c!--\r\nInformation about the configuration if relevant, e.g. basic network, advanced networking, etc.  N/A otherwise\r\n--\u003e\r\n\r\n\r\n##### OS / ENVIRONMENT\r\nUsed Ubuntu 18:04 as a host for management server and KVM Hypervisor.\r\n\r\n##### SUMMARY\r\n\r\nHello Cloudstack community. There is a systemvm remote logging feature proposal. It enabled administrator to access systemvm's log file through management server. I think it would be interesting for other as well. To better explaining my setup I depicted the configuration in below picture.\r\n\r\n![Cloudstack Centralized logging-2](https://user-images.githubusercontent.com/9499410/82427752-cdabb200-9a89-11ea-8e8e-c4eb0744076f.png)\r\n\r\nIn this case rather than downloading log file manually from systemvms, they will be available automatically through management server.\r\n\r\n### STEPS TO REPRODUCE\r\n\r\nBelow are configuration files for hypervisor, management server and sytemvms.\r\n\r\n#### Hypervisors\r\n\r\nBelow 4 lines have to uncommented in rsyslog configuration file on each hypervisor in `/etc/rsyslog.conf`\r\n\r\n```Python\r\nmodule(load=\"imtcp\")\r\ninput(type=\"imtcp\" port=\"514\")\r\n\r\nmodule(load=\"imudp\")\r\ninput(type=\"imudp\" port=\"514\")\r\n```\r\n\r\nAdding below file on each hypervisor to export logs from hypervisor to management server.\r\n\r\n```Java\r\n# cat /etc/rsyslog.d/00-fwd.conf\r\nif ($fromhost-ip startswith '169.') then {\r\n    *.* @\u003cManagement Server IP\u003e:514\r\n    stop\r\n}\r\n```\r\n\r\n#### Management Server\r\n\r\nJust like hypervisors, below 4 lines have to uncommented in rsyslog configuration file on management server in `/etc/rsyslog.conf`\r\n\r\n```Python\r\nmodule(load=\"imtcp\")\r\ninput(type=\"imtcp\" port=\"514\")\r\n\r\nmodule(load=\"imudp\")\r\ninput(type=\"imudp\" port=\"514\")\r\n```\r\n\r\nAdd below file on management server\r\n\r\n```Java\r\n# cat /etc/rsyslog.d/00-remote.conf\r\n$template remote-incoming-logs,\"/var/log/rsyslog/%HOSTNAME%/syslog\"\r\n$template remote-incoming-logs-combined,\"/var/log/rsyslog/remote\"\r\nif ($fromhost-ip startswith '10.' or $fromhost-ip startswith '172.') then ?remote-incoming-logs\r\nif ($fromhost-ip startswith '10.' or $fromhost-ip startswith '172.') then ?remote-incoming-logs-combined\r\n\u0026 ~ \r\n```\r\n\r\n#### Systemvm\r\n\r\n`00-fwd.conf` file should add on each systemvm as well to export logs from systemvms to hypervisors.\r\n\r\n```Java\r\n# cat /etc/rsyslog.d/00-fwd.conf\r\n*.*  @169.254.0.1:514\r\n```\r\n\r\n`49-cloud.conf` file should add on each systemvm. This file describes the logs that eventually sent to a management server\r\n\r\n```Java\r\n # cat /etc/rsyslog.d/49-cloud.conf\r\n$InputFileName /var/log/cloud.log\r\n$InputFileTag cloud\r\n$InputFileStateFile stat-file1-cloud\r\n$InputFileSeverity info \r\n$InputFileFacility local7 \r\n$InputFilePollInterval 1 \r\n$InputFilePersistStateInterval 1 \r\n$InputRunFileMonitor\r\n```\r\n \r\nNeed to uncomment below lines if they are commented in /etc/rsyslog.conf\r\n\r\n```python\r\n$ModLoad imudp\r\n$UDPServerRun 3914\r\n```\r\n\r\n##### EXPECTED RESULTS\r\n\r\n~~~\r\nManagement server store systemvm's logs in /var/log/rsyslog/\u003cSystemvm's host name\u003e/syslog\r\n~~~\r\n","author":{"url":"https://github.com/soreana","@type":"Person","name":"soreana"},"datePublished":"2020-05-20T10:24:52.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":13},"url":"https://github.com/4093/cloudstack/issues/4093"}