T205849: Begin the implementation of Q1's Logging Infrastructure design (2018-19 Q2 Goal)

Phabricator Link	T205849
Status	resolved
Priority	Medium (orange)
Points
Phabricator Task Created	2018/10/01 08:53 AM
Wiki Page Created	2019/06/09 01:27 PM
Phabricator Task Last Modified	2019/08/19 11:16 AM
Wiki Page Last Updated	2021/02/16 12:21 PM
Phabricator Task Closed	2019/01/16 05:25 AM
Authored By	fgiunchedi (Filippo Giunchedi)
Assigned To	fgiunchedi (Filippo Giunchedi)
Projects	Platform Team Legacy (Watching / External) SRE (Backlog) Services (watching) User-fgiunchedi (Doing) User-herron (Working on) Wikimedia-Logstash (In Dev/Progress) observability (Inbox)
Platform Engineering Initiative Column
Platform Team Workboards Column
Subtasks	T206454: Setup Kafka cluster, producers and consumers for logging pipeline (resolved) T205873: Investigate Kafka main cluster usage for logging pipeline (resolved) T203169: Logstash hardware expansion (resolved) T206633: Setup rsyslog to be able to produce logs to Kafka (resolved) T205855: Investigate approaches to ingest sensitive log producers (resolved) T205852: Onboard at least 10 new non-sensitive log producers to the logging pipeline (resolved) T205851: Migrate >=90% of existing Logstash traffic to the logging pipeline (resolved) T205850: Procure and provision Logging pipeline hardware in multiple datacenters (resolved)
Parent Tasks

Column Transitions: (Added project: ⇑, Removed project: ⇓,Entered column: ⇒, Exited column: ⇐)

Project	Column	Date
⇑ SRE		1 October 2018
⇑ Wikimedia-Logstash		1 October 2018
⇑ User-fgiunchedi		3 October 2018
User-fgiunchedi	⇐ Backlog	3 October 2018
User-fgiunchedi	⇒ Doing	3 October 2018
Wikimedia-Logstash	⇐ Backlog	15 October 2018
Wikimedia-Logstash	⇒ In Dev/Progress	15 October 2018
⇑ Services (watching)		30 October 2018
⇑ Platform Team Legacy (Watching / External)		30 October 2018
⇑ Patch-For-Review		5 November 2018
⇑ User-herron		12 November 2018
User-herron	⇐ Backlog	19 November 2018
User-herron	⇒ Working on	19 November 2018
⇑ observability		19 August 2019
⇓ Patch-For-Review		19 August 2019