Tasks By Assignee

Assignee: fgiunchedi (Filippo Giunchedi)

Switch to Active Tasks 52 Phabricator task(s).

Phabricator Link	Wiki Link	Status	Priority	Author	Assignee	Projects	Subtasks	Parent Tasks
T102015	T102015: put new restbase servers in service	resolved	Medium (orange)	fgiunchedi (Filippo Giunchedi)	fgiunchedi (Filippo Giunchedi)	RESTBase-Cassandra (Backlog) RESTBase-architecture (Backlog) Patch-For-Review Services (Backlog) SRE (Backlog)	T104208: alternative Cassandra metrics reporting T102557: investigate new restbase machine disks timeouts	T93790: Expand RESTBase cluster capacity
T102575	T102575: document graphite failover/backfill procedures	resolved	Medium (orange)	fgiunchedi (Filippo Giunchedi)	fgiunchedi (Filippo Giunchedi)	Documentation (New Technical Documentation Request) SRE Observability (Backlog) SRE (Acknowledged) observability (Backlog) Graphite (Inbox)		T85451: scale graphite deployment (tracking)
T104208	T104208: alternative Cassandra metrics reporting	resolved	Medium (orange)	Eevans (Eric Evans)	fgiunchedi (Filippo Giunchedi)	RESTBase-architecture (Backlog) Patch-For-Review Services (Backlog) SRE (Backlog) RESTBase-Cassandra (Backlog)		T102015: put new restbase servers in service
T105573	T105573: Automate grafana dashboard backups	resolved	Medium (orange)	GWicke (Gabriel Wicke)	fgiunchedi (Filippo Giunchedi)	Graphite (Inbox) Platform Team Legacy (Watching / External) Services (watching)
T113733	T113733: column family cassandra metrics size	resolved	Medium (orange)	fgiunchedi (Filippo Giunchedi)	fgiunchedi (Filippo Giunchedi)	RESTBase (Backlog) SRE (Backlog) Patch-For-Review RESTBase-Cassandra (Backlog) Services (watching)		T95253: Finish conversion to multiple Cassandra instances per hardware node
T113939	T113939: assess impact of many cassandra seed nodes with multi instance	resolved	High (red)	fgiunchedi (Filippo Giunchedi)	fgiunchedi (Filippo Giunchedi)	RESTBase (Backlog) SRE (Backlog) RESTBase-Cassandra (Backlog) Patch-For-Review		T95253: Finish conversion to multiple Cassandra instances per hardware node
T119651	T119651: Set $wmfSwiftCodfwConfig in PrivateSettings	resolved	Medium (orange)	aaron (Aaron Schulz)	fgiunchedi (Filippo Giunchedi)	SRE (Backlog) Sustainability (Tag)		T91869: Implement a replication strategy for Swift
T127455	T127455: Enable HTTPS for Swift traffic	resolved	Medium (orange)	faidon (Faidon Liambotis)	fgiunchedi (Filippo Giunchedi)	Sustainability (MediaWiki-MultiDC) (Later) SRE-swift-storage (Inbox) Patch-For-Review SRE (Backlog)		T88445: MediaWiki active/active datacenter investigation and work (tracking)
T130938	T130938: rack/setup new host graphite2002	resolved	High (red)	RobH (Rob Halsell)	fgiunchedi (Filippo Giunchedi)	Patch-For-Review ops-codfw (Backlog) SRE (Backlog)		T126253: additional graphite machines request, 1x per DC
T131253	T131253: Report ok / broken metrics from service_checker	resolved	Medium (orange)	GWicke (Gabriel Wicke)	fgiunchedi (Filippo Giunchedi)	Platform Team Legacy (Later) Services (later) observability (Inbox)
T132717	T132717: rack and set up graphite1003	resolved	Medium (orange)	Cmjohnson (cmjohnson)	fgiunchedi (Filippo Giunchedi)	Patch-For-Review SRE (Backlog)		T126253: additional graphite machines request, 1x per DC
T134889	T134889: put additional graphite machines in service	resolved	High (red)	fgiunchedi (Filippo Giunchedi)	fgiunchedi (Filippo Giunchedi)	Graphite (Carbon / Whisper) Patch-For-Review SRE (Backlog)	T135385: investigate carbon-c-relay stalls/drops towards graphite2002	T85451: scale graphite deployment (tracking)
T135385	T135385: investigate carbon-c-relay stalls/drops towards graphite2002	declined	High (red)	fgiunchedi (Filippo Giunchedi)	fgiunchedi (Filippo Giunchedi)	Graphite (Carbon / Whisper) SRE (Acknowledged)		T134889: put additional graphite machines in service
T181964	T181964: Clean metrics for restbase erroneus legacy tables from cassandra 3 cluster	resolved	High (red)	fgiunchedi (Filippo Giunchedi)	fgiunchedi (Filippo Giunchedi)	Cassandra (Backlog) User-Eevans (Blocked) RESTBase (Backlog) Services (doing)		T183745: FY17/18 Q3 Program 7 Services Goal: Full migration to Cassandra 3
T186562	T186562: Reimage JBO-RAID0 configured RESTBase HP machines	resolved	Medium (orange)	Eevans (Eric Evans)	fgiunchedi (Filippo Giunchedi)	Cassandra (Backlog) User-Eevans (Backlog) Services (doing) Platform Team Workboards (Done with CPT)		T178177: Investigate aberrant Cassandra columnfamily read latency of restbase101{0,2,4} T189057: Understand (and if possible, improve) cluster performance under the new storage strategy
T196067	T196067: Clean up cpjobqueue metrics	resolved	Medium (orange)	Pchelolo	fgiunchedi (Filippo Giunchedi)	Services (watching) Event-Platform (Backlog) Analytics (Analytics-Radar) (Radar - Watching) Graphite (Inbox) User-fgiunchedi (Up next) SRE (Backlog) MediaWiki-Core-JobQueue		T190327: FY17/18 Q4 Program 8 Services Goal: Complete the JobQueue transition to EventBus
T199479	T199479: Add alerts for Logstash rates in production	resolved	Medium (orange)	Krinkle (Timo Tijhof)	fgiunchedi (Filippo Giunchedi)	Platform Team Legacy (Watching / External) SRE (Acknowledged) observability (Inbox) Sustainability (Incident Followup)
T203169	T203169: Logstash hardware expansion	resolved	Medium (orange)	fgiunchedi (Filippo Giunchedi)	fgiunchedi (Filippo Giunchedi)	User-herron (Working on) User-fgiunchedi (Other) Wikimedia-Logstash (Externally blocked) observability (Inbox) SRE (Backlog)	T211065: rack/setup/install codfw logstash elasticsearch storage servers T211859: cronspam from elasticsearch-curator on stretch	T205849: Begin the implementation of Q1's Logging Infrastructure design (2018-19 Q2 Goal) T205873: Investigate Kafka main cluster usage for logging pipeline
T204245	T204245: Run MediaWiki media originals active/active	open	Medium (orange)	fgiunchedi (Filippo Giunchedi)	fgiunchedi (Filippo Giunchedi)	Performance-Team (Radar) (Watching) Epic Services (watching) User-fgiunchedi (Radar) Sustainability (MediaWiki-MultiDC) (Discuss next)		T88445: MediaWiki active/active datacenter investigation and work (tracking)
T205849	T205849: Begin the implementation of Q1's Logging Infrastructure design (2018-19 Q2 Goal)	resolved	Medium (orange)	fgiunchedi (Filippo Giunchedi)	fgiunchedi (Filippo Giunchedi)	Platform Team Legacy (Watching / External) Services (watching) SRE (Backlog) observability (Inbox) User-herron (Working on) Wikimedia-Logstash (In Dev/Progress) User-fgiunchedi (Doing)	T206454: Setup Kafka cluster, producers and consumers for logging pipeline T205873: Investigate Kafka main cluster usage for logging pipeline T203169: Logstash hardware expansion T206633: Setup rsyslog to be able to produce logs to Kafka T205855: Investigate approaches to ingest sensitive log producers T205852: Onboard at least 10 new non-sensitive log producers to the logging pipeline T205851: Migrate >=90% of existing Logstash traffic to the logging pipeline T205850: Procure and provision Logging pipeline hardware in multiple datacenters
T205851	T205851: Migrate >=90% of existing Logstash traffic to the logging pipeline	resolved	Medium (orange)	fgiunchedi (Filippo Giunchedi)	fgiunchedi (Filippo Giunchedi)	Wikimedia-Logstash (In Dev/Progress) observability (Inbox) SRE (Backlog)	T211124: Move mediawiki to new logging infrastructure	T205849: Begin the implementation of Q1's Logging Infrastructure design (2018-19 Q2 Goal)
T205852	T205852: Onboard at least 10 new non-sensitive log producers to the logging pipeline	resolved	Medium (orange)	fgiunchedi (Filippo Giunchedi)	fgiunchedi (Filippo Giunchedi)	Patch-For-Review observability (Inbox) SRE (Backlog) Wikimedia-Logstash (Up next) User-herron (Backlog)	T210846: Ship Grafana server logs to ELK T210458: Ship PuppetDB logs to ELK T210455: Ship prometheus logs to ELK T209860: Ship peopleweb apache2 error logs to ELK T143733: Send Jenkins daemon logs to logstash T141895: Get phabricator error logs into logstash T141324: Look into shoving gerrit logs into logstash T63789: Add Zookeeper log files to logstash T63788: Add kafka log files to logstash T63780: Add swift logs to logstash T7: Get icinga alerts into logstash	T205849: Begin the implementation of Q1's Logging Infrastructure design (2018-19 Q2 Goal)
T205855	T205855: Investigate approaches to ingest sensitive log producers	resolved	Medium (orange)	fgiunchedi (Filippo Giunchedi)	fgiunchedi (Filippo Giunchedi)	Wikimedia-Logstash (Up next) observability (Inbox) SRE (Backlog)	T197173: Ship MX logs to ELK T193766: Ship host syslogs to ELK	T205849: Begin the implementation of Q1's Logging Infrastructure design (2018-19 Q2 Goal)
T205873	T205873: Investigate Kafka main cluster usage for logging pipeline	resolved	High (red)	fgiunchedi (Filippo Giunchedi)	fgiunchedi (Filippo Giunchedi)	observability (Inbox) SRE (Backlog) Wikimedia-Logstash (Backlog)	T203169: Logstash hardware expansion	T205849: Begin the implementation of Q1's Logging Infrastructure design (2018-19 Q2 Goal)
T206633	T206633: Setup rsyslog to be able to produce logs to Kafka	resolved	Medium (orange)	fgiunchedi (Filippo Giunchedi)	fgiunchedi (Filippo Giunchedi)	SRE (Backlog) User-fgiunchedi (Doing) Wikimedia-Logstash (In Dev/Progress) observability (Inbox) SRE Observability (Inbox) Patch-For-Review		T205849: Begin the implementation of Q1's Logging Infrastructure design (2018-19 Q2 Goal)
T209136	T209136: python3-etcd needs python3-dnspython	resolved	Medium (orange)	ema (Emanuele Rocca)	fgiunchedi (Filippo Giunchedi)	Services (watching) SRE (Backlog) Patch-For-Review Platform Team Workboards (Done with CPT) SRE-tools (Backlog)
T211124	T211124: Move mediawiki to new logging infrastructure	resolved	Medium (orange)	fgiunchedi (Filippo Giunchedi)	fgiunchedi (Filippo Giunchedi)	MediaWiki-Debug-Logger SRE (Backlog) Wikimedia-Logstash (Backlog) observability (Inbox)		T205851: Migrate >=90% of existing Logstash traffic to the logging pipeline
T211416	T211416: Put restbase201[3-8] into conftool and LVS	resolved	High (red)	mobrovac (Marko Obrovac)	fgiunchedi (Filippo Giunchedi)	User-fgiunchedi (Backlog) SRE (Backlog) Platform Team Workboards (Done with CPT) Services (done)
T219523	T219523: Session storage Cassandra metrics (Prometheus) not being collected	resolved	Needs Triage (violet)	Eevans (Eric Evans)	fgiunchedi (Filippo Giunchedi)	SRE (Backlog) User-fgiunchedi (Backlog) Patch-For-Review		T215883: Session storage Cassandra cluster configuration
T219924	T219924: Move mobileapps logging to new logging pipeline	resolved	Medium (orange)	Pchelolo	fgiunchedi (Filippo Giunchedi)	Page Content Service (Backlog) Wikimedia-Logstash (Backlog) SRE (Backlog) Platform Team Legacy (Watching / External) service-runner (Backlog) observability (Externally blocked) Services (watching) Product-Infrastructure-Team-Backlog (Backlog)	T239459: service-runner apps running on kubernetes emit logs with log level 50	T211125: Move service-runner to new logging infrastructure
T219925	T219925: Move proton logging to new logging pipeline	resolved	Medium (orange)	Pchelolo	fgiunchedi (Filippo Giunchedi)	Platform Team Legacy (Watching / External) Proton Wikimedia-Logstash (Backlog) Services (watching) service-runner (Backlog) SRE (Backlog) Product-Infrastructure-Team-Backlog (Backlog) observability (Externally blocked)	T239459: service-runner apps running on kubernetes emit logs with log level 50	T211125: Move service-runner to new logging infrastructure
T234564	T234564: Logstash discards messages from MediaWiki if they contain uncommon keys in the $context array	resolved	High (red)	matmarex (Bartosz Dziewoński)	fgiunchedi (Filippo Giunchedi)	Release-Engineering-Team-TODO (201910) (INBOX) Performance-Team (Radar) (Watching) MW-1.35-notes (1.35.0-wmf.4; 2019-10-29) Wikimedia-production-error (Untriaged) VisualEditor (External and Administrivia) Deployments (To Triage) Patch-For-Review User-Ryasmeen (To be Verified) Wikimedia-Logstash (Backlog) Release-Engineering-Team (Deployment services)
T76986	T76986: RESTBase production hardware	resolved	High (red)	GWicke (Gabriel Wicke)	fgiunchedi (Filippo Giunchedi)	RESTBase-architecture (Backlog) RESTBase (Blocked / others) hardware-requests (Backlog) Blocked-on-Operations (Backlog) SRE (Backlog)	T89639: restbase1006 faulty disk controller T88805: rack and setup restbase production cluster in eqiad	T1228: RESTbase deployment
T78514	T78514: Detailed cassandra monitoring: metrics and dashboards done, need to set up alerts	resolved	Medium (orange)	GWicke (Gabriel Wicke)	fgiunchedi (Filippo Giunchedi)	Patch-For-Review RESTBase (In progress) SRE (Backlog) observability (Inbox)	T93886: Cassandra/CQL query interface monitoring T97024: some cassandra metrics sent with invalid values T93884: graphs for Cassandra metrics	T89481: RESTBase beta release (revision storage / content API)
T85451	T85451: scale graphite deployment (tracking)	invalid	High (red)	fgiunchedi (Filippo Giunchedi)	fgiunchedi (Filippo Giunchedi)	Tracking-Neverending (Tag) Services (watching) Platform Team Legacy (Watching / External) Patch-For-Review SRE (Acknowledged) Graphite (Inbox) WMDE-Analytics-Engineering (Watching / External)	T134889: put additional graphite machines in service T90111: replace txstatsd T85909: migrate graphite to new hardware T85908: replicate metric traffic in eqiad and codfw T85907: acquire graphite hardware in codfw and eqiad T126253: additional graphite machines request, 1x per DC T102575: document graphite failover/backfill procedures T90591: backfill metrics from tungsten to graphite1001 T89857: scale statsd reporting/aggregation (plan) T86316: graphite clustering plan
T85907	T85907: acquire graphite hardware in codfw and eqiad	resolved	High (red)	fgiunchedi (Filippo Giunchedi)	fgiunchedi (Filippo Giunchedi)	SRE (Backlog) Graphite (Inbox)		T85451: scale graphite deployment (tracking)
T85908	T85908: replicate metric traffic in eqiad and codfw	resolved	High (red)	fgiunchedi (Filippo Giunchedi)	fgiunchedi (Filippo Giunchedi)	SRE (Backlog) Graphite (Inbox) Patch-For-Review		T85451: scale graphite deployment (tracking)
T85909	T85909: migrate graphite to new hardware	resolved	High (red)	fgiunchedi (Filippo Giunchedi)	fgiunchedi (Filippo Giunchedi)	Graphite (Inbox) SRE (Backlog) Patch-For-Review	T86555: set graphite2001 asset tag mgmt entries	T85451: scale graphite deployment (tracking)
T86316	T86316: graphite clustering plan	declined	Medium (orange)	fgiunchedi (Filippo Giunchedi)	fgiunchedi (Filippo Giunchedi)	Graphite (Inbox) SRE (Acknowledged)		T85451: scale graphite deployment (tracking)
T88805	T88805: rack and setup restbase production cluster in eqiad	resolved	High (red)	Cmjohnson (cmjohnson)	fgiunchedi (Filippo Giunchedi)	SRE (Backlog) ops-eqiad (Backlog)	T83551: Split memcached in eqiad across multiple racks/rows	T76986: RESTBase production hardware
T89366	T89366: Access to restbase / cassandra cluster	resolved	High (red)	GWicke (Gabriel Wicke)	fgiunchedi (Filippo Giunchedi)	SRE-Access-Requests (Untriaged) SRE (Backlog) Blocked-on-Operations (Backlog) Patch-For-Review RESTBase (Blocked / others)		T1228: RESTbase deployment T89567: Load test production cluster
T89636	T89636: setup internal LVS for restbase eqiad servers	resolved	High (red)	fgiunchedi (Filippo Giunchedi)	fgiunchedi (Filippo Giunchedi)	RESTBase (Backlog) Patch-For-Review Services (Backlog) SRE (Backlog)		T1228: RESTbase deployment
T89639	T89639: restbase1006 faulty disk controller	resolved	High (red)	fgiunchedi (Filippo Giunchedi)	fgiunchedi (Filippo Giunchedi)	SRE (Backlog) Services (Backlog) ops-eqiad (Backlog) RESTBase (Backlog)		T1228: RESTbase deployment T76986: RESTBase production hardware
T89857	T89857: scale statsd reporting/aggregation (plan)	invalid	Medium (orange)	fgiunchedi (Filippo Giunchedi)	fgiunchedi (Filippo Giunchedi)	Graphite (statsd) SRE (Acknowledged)		T85451: scale graphite deployment (tracking)
T90111	T90111: replace txstatsd	resolved	High (red)	fgiunchedi (Filippo Giunchedi)	fgiunchedi (Filippo Giunchedi)	SRE (Backlog) Graphite (Inbox) Patch-For-Review	T95703: enable statsd extended counters T95627: Counts with underscore in name no longer updated since move to statsite (cassandra metrics)	T85451: scale graphite deployment (tracking)
T90591	T90591: backfill metrics from tungsten to graphite1001	resolved	Medium (orange)	fgiunchedi (Filippo Giunchedi)	fgiunchedi (Filippo Giunchedi)	SRE (Backlog) Patch-For-Review Graphite (Inbox)		T85451: scale graphite deployment (tracking)
T93886	T93886: Cassandra/CQL query interface monitoring	resolved	Needs Triage (violet)	Eevans (Eric Evans)	fgiunchedi (Filippo Giunchedi)	RESTBase (Backlog) Patch-For-Review SRE (Backlog) RESTBase-Cassandra (Backlog)		T78514: Detailed cassandra monitoring: metrics and dashboards done, need to set up alerts
T94741	T94741: Update Cassandra package in Jessie to 2.1.4	resolved	High (red)	GWicke (Gabriel Wicke)	fgiunchedi (Filippo Giunchedi)	RESTBase (Backlog) RESTBase-Cassandra (Backlog) SRE (Backlog)		T92471: enable authenticated access to Cassandra JMX
T95253	T95253: Finish conversion to multiple Cassandra instances per hardware node	resolved	High (red)	GWicke (Gabriel Wicke)	fgiunchedi (Filippo Giunchedi)	RESTBase-Cassandra (Backlog) SRE (Backlog) RESTBase (Under discussion) Cassandra (In-Progress) Blocked-on-Operations (Backlog) Patch-For-Review	T130540: Figure out if nodes in different DCs can be bootstrapped in parallel T121535: Perform cleanups to reclaim space from recent topology changes T117114: Ensure ansible-deploy can cope with multi-instance restarts T113939: assess impact of many cassandra seed nodes with multi instance T113733: column family cassandra metrics size	T93790: Expand RESTBase cluster capacity
T95627	T95627: Counts with underscore in name no longer updated since move to statsite (cassandra metrics)	resolved	High (red)	GWicke (Gabriel Wicke)	fgiunchedi (Filippo Giunchedi)	SRE (Backlog) Graphite (Inbox)		T90111: replace txstatsd
T95703	T95703: enable statsd extended counters	resolved	High (red)	GWicke (Gabriel Wicke)	fgiunchedi (Filippo Giunchedi)	Graphite (Inbox) Patch-For-Review SRE (Backlog)		T90111: replace txstatsd
T97024	T97024: some cassandra metrics sent with invalid values	resolved	Medium (orange)	fgiunchedi (Filippo Giunchedi)	fgiunchedi (Filippo Giunchedi)	RESTBase-Cassandra (Backlog) observability (Inbox) Patch-For-Review		T78514: Detailed cassandra monitoring: metrics and dashboards done, need to set up alerts