{"id":348,"date":"2019-04-12T21:10:02","date_gmt":"2019-04-12T21:10:02","guid":{"rendered":"https:\/\/bootstrap-it.com\/blog\/?p=348"},"modified":"2019-04-12T21:10:02","modified_gmt":"2019-04-12T21:10:02","slug":"cleaning-up-after-spectre-and-meltdown-figuring-out-how-badly-they-slowed-down-your-servers","status":"publish","type":"post","link":"https:\/\/bootstrap-it.com\/blog\/?p=348","title":{"rendered":"Cleaning up after Spectre and Meltdown: figuring out how badly they slowed down your servers"},"content":{"rendered":"
<\/a><\/a><\/a><\/a><\/a><\/div>\n

This article is based on my <\/em>Linux Performance Monitoring and Tuning course on Pluralsight<\/em><\/a> which, by coincidence, was released just before news of Spectre and Meltdown broke.<\/em><\/p>\n\n\n\n

By now the critical stage of the Spectre and Meltdown vulnerabilities<\/a> is largely over. Patches for affected operating systems have been written and, assuming you keep your servers properly updated, you should be safe.<\/p>\n\n\n\n

That\u2019s the easy part. The real problem is that the patch might slow your system down\u200a\u2014\u200aparticularly if you\u2019re running applications that interact often with the kernel. So you\u2019ll want to know just how much of a hit you\u2019ve taken, and what upgrades you\u2019ll need to get you back to where you should be.<\/p>\n\n\n\n

My recent Pluralsight course<\/a> discussed understanding, diagnosing, tuning, and monitoring performance on Linux servers. Those are the tools of choice for the kind of assessment and administration this problem demands, so I thought I\u2019d make this guide to installing and using the collectd daemon to gather and monitor performance metrics locally or across a network.<\/p>\n\n\n\n

Besides collectd, the course also covers the Nagios, Munin, and nmon monitoring tools, along with analysing metrics and optimising existing stacks.<\/p>\n\n\n\n

Ideally, you would have some older, pre-Spectre, performance data available to use as a baseline. But even if you have to start from scratch, it won\u2019t take too long for collectd to start pointing fingers at your pain points.<\/p>\n\n\n\n

Installing collectd<\/h4>\n\n\n\n

collectd is a Linux system daemon that collects and organizes system performance data into Round Robin Database (RRD) files normally kept in the \/var\/lib\/collectd\/rrd\/ directory tree.<\/p>\n\n\n\n

I\u2019m going to take you through the process of installing and configuring collectd. I\u2019ll then demo a tool called collectd-web that will convert the RRD data into graphs. Once that\u2019s all done, I\u2019ll show you how to set up a couple of remote servers as collectd clients so we can access data from the entire fleet on a single server. But one thing at a time.<\/p>\n\n\n\n

Installing collectd as a standalone is simple: you need nothing more than the collectd package. Getting the collectd-web tool installed and running, on the other hand, can be a bit of a trick. Since clients require nothing more than collectd, I\u2019ll focus on setting up the machine our collectd server. I\u2019m going to be doing this on an LXC container running Ubuntu 16.04.<\/p>\n\n\n\n

Here\u2019s the package list.<\/p>\n\n\n\n

# apt install -y collectd python build-essential \\
 librrds-perl librrds-perl libjson-perl libhtml-parser-perl \\
 apache2<\/pre>\n\n\n\n
Let me break that down for you. We\u2019ll need Python to run the collectd-web tool and build-essential to build the source code we\u2019ll get later. Those Perl libraries are to interpret the RRD database files and to add JSON and CGI functionality. Finally, we\u2019ll be using Apache as our web server to handle the web requests themselves. I\u2019ll let all that happen by itself and skip the boring details. 
 
Next I\u2019ll use a2enmod to enable the CGI modules that\u2019ll allow Apache to manage some of the visualizations.<\/p>\n\n\n\n
# a2enmod cgi cgid<\/pre>\n\n\n\n
cpan will install a couple of Perl modules. Make sure you use uppercase for JSON and CGI.<\/p>\n\n\n\n
# cpan JSON
# cpan CGI<\/pre>\n\n\n\n
I\u2019m going to use a text editor to add a section to the 000-default.conf file in the \/etc\/apache2\/sites-available\/ directory that will enable Apache CGI script handling from the web server end.<\/p>\n\n\n\n
<Directory \/var\/www\/html\/collectd-web\/cgi-bin>
Options Indexes ExecCGI
AllowOverride All
AddHandler cgi-script .cgi
Require all granted
<\/Directory><\/pre>\n\n\n\n
With that done, there\u2019ll be only one more thing to do get Apache just the way we want it. Take a look at the apache2.conf configuration file and scroll down to the Include line where we set the network ports on which to listen. Note how it\u2019s all offloaded to a file called ports.conf.<\/p>\n\n\n\n
# Include list of ports to listen on
Include ports.conf<\/pre>\n\n\n\n
If we want to be able to access our data remotely through a browser, that\u2019s the file we want to edit. I\u2019ll just make sure the \u201cListen\u201d line in ports.conf reads 0.0.0.0:80, so that we\u2019re not restricted to only local guests.<\/p>\n\n\n\n
Listen 0.0.0.0:80<\/pre>\n\n\n\n
With all our Apache work done, now\u2019s the time to restart it so everything is properly updated.<\/p>\n\n\n\n
# systemctl restart apache2<\/pre>\n\n\n\n
Now we should take a quick look at the collectd.conf configuration file in the \/etc\/collectd\/ directory. It\u2019s a big file and most of the default settings will work for us right now, but you should be aware of the kinds of things you can add and how they\u2019re configured. Most of collectd\u2019s power is delivered through its plugins, which you load (by uncommenting) in the LoadPlugin section. Some of the plugins might already be loaded, like apache and cpu near the top.<\/p>\n\n\n\n
LoadPlugin apache
#LoadPlugin apcups
#LoadPlugin ascent
[\u2026]
LoadPlugin cpu<\/pre>\n\n\n\n
This means that, when properly configured below, collectd will monitor and report Apache and CPU behavior.<\/p>\n\n\n\n
You\u2019ll also need to configure each of your plugins below in the configuration section. The cpu configuration is commented out, but it will work fine with its default values. In this case, the configuration section is only for people who might want to customize it.<\/p>\n\n\n\n
#<Plugin cpu>
# ReportByCpu true
# ReportByState true
# ValuesPercentage false
#<\/Plugin><\/pre>\n\n\n\n
If you do edit the config file, be sure to restart the daemon using systemctl. This is also a good place to look for error messages in case something went wrong. Pay close attention to any error messages you get\u200a\u2014\u200aespecially through the system logs\u200a\u2014\u200aand carefully check the syntax you used in the configuration file. It\u2019s also possible that you\u2019re missing a Perl module or two.<\/p>\n\n\n\n
# systemctl restart collectd
# systemctl status collectd<\/pre>\n\n\n\n
But before all this will be useful, we\u2019ll need to install the collectd-web tool itself. I\u2019ll first install git, and then clone the collectd-web repository. This will save the source code to a new directory called collectd-web.<\/p>\n\n\n\n
# apt install git
$ git clone https:\/\/github.com\/httpdss\/collectd-web.git<\/a><\/pre>\n\n\n\n
There\u2019s a script in the cgi-bin subdirectory that we\u2019ll have to make executable, so I\u2019ll change to cgi-bin and use chmod to add an executable bit to graphdefs.cgi. Then I\u2019ll head back one level to the collectd-web directory so I can make one edit to the Python runserver script.<\/p>\n\n\n\n
$ cd collectd-web\/cgi-bin
$ chmod +x graphdefs.cgi
$ cd ..
$ nano runserver.py<\/pre>\n\n\n\n
I\u2019m going to change the two places where the localhost is referenced by 127.0.0.1, and open them up to the entire internet by editing them to read 0.0.0.0. This will allow collectd to listen for data from other machines.<\/p>\n\n\n\n
With that, we\u2019re all set. Make sure you\u2019re in the collectd-web directory, and run the runserver.py script. I\u2019ll add the ampersand character so it\u2019ll run in the background and I\u2019ll get my command line back. It\u2019s a good idea to keep an eye on this shell, however, since events are printed here that can sometimes be useful.<\/p>\n\n\n\n
$ .\/runserver.py &<\/pre>\n\n\n\n
Now I\u2019ll fire up a browser and point it to the LXC container\u2019s IP address, specifying the port 8888. Speaking of network ports, if you\u2019re running a firewall, you\u2019ll need to open up ports 8888 and 25826. You can click on the name of your host and then select from the metrics displayed below.<\/p>\n\n\n\n
\"\"\/
The CPU display for the local machine (called collectd)<\/figcaption><\/figure>\n\n\n\n
A multi-host collectd configuration<\/h4>\n\n\n\n
If you\u2019ve been following along with the demo so far you\u2019ve got metrics from a single server being displayed through a local web service. But if you\u2019ve got lots of servers, you probably won\u2019t enjoy opening a new browser tab for each one\u200a\u2014\u200aespecially not if you\u2019re expected to cycle through them periodically through your work day.<\/p>\n\n\n\n
So why not send the data to a single server? Let\u2019s do that. I\u2019ve launched two servers and installed nothing more than the collectd package on each. Their hostnames are collectd-client1 and collectd-client2\u200a\u2014\u200awhich will make it easier to identify them later.<\/p>\n\n\n\n
I\u2019m logged into one of my clients and I\u2019m going to edit the collectd.conf configuration file. At the very least I\u2019ll need to make sure the network plugin is enabled and then head down to the configuration to set things up. All I need to do there is uncomment the open and close Plugin network lines, the open and close Server lines, and then edit the Server IP address to match IP used by my Collectd server. This is so the client can push data to the server. I\u2019ll leave the other lines at their default values.<\/p>\n\n\n\n
LoadPlugin network<\/pre>\n\n\n\n
<Plugin network>
# # client setup:
# Server \u201cff18::efc0:4a42\u201d \u201c25826\u201d
 <Server \u201c10.0.3.102\u201d \u201c25826\u201d>
# SecurityLevel Encrypt
# Username \u201cuser\u201d
# Password \u201csecret\u201d
# Interface \u201ceth0\u201d
# ResolveInterval 14400
 <\/Server>
# TimeToLive 128
#
# # server setup:
# Listen \u201cff18::efc0:4a42\u201d \u201c25826\u201d
# <Listen \u201c239.192.74.66\u201d \u201c25826\u201d>
# SecurityLevel Sign
# AuthFile \u201c\/etc\/collectd\/passwd\u201d
# Interface \u201ceth0\u201d
# <\/Listen>
# MaxPacketSize 1452
#
# # proxy setup (client and server as above):
# Forward true
#
# # statistics about the network plugin itself
# ReportStats false
#
# # \u201cgarbage collection\u201d
# CacheFlush 1800
<\/Plugin><\/pre>\n\n\n\n
That\u2019s it. You\u2019ll restart the collectd service and then repeat the process on any other clients you want to monitor.<\/p>\n\n\n\n
# systemctl restart collectd<\/pre>\n\n\n\n
Now, back on the server, I\u2019ll edit the same configuration file, and make sure that networking is enabled. Again, as before, I\u2019ll scroll down to the networking configuration section and uncomment the Plugin lines, but this time, I\u2019ll uncomment and edit the Server Setup section rather than the Client section. Here, I\u2019ll change the Listen line to 0.0.0.0 so the server will listen for incoming data from anywhere.<\/p>\n\n\n\n
# server setup:
# Listen \u201cff18::efc0:4a42\u201d \u201c25826\u201d
 <Listen \u201c0.0.0.0\u201d \u201c25826\u201d>
# SecurityLevel Sign
# AuthFile \u201c\/etc\/collectd\/passwd\u201d
# Interface \u201ceth0\u201d
 <\/Listen><\/pre>\n\n\n\n
I\u2019ll save and close the file and restart both the collectd and apache services.<\/p>\n\n\n\n
# systemctl restart collectd<\/pre>\n\n\n\n
Then, making sure that the Python runserver script isn\u2019t already running\u200a\u2014\u200aif it is you should shut it down using kill\u200a\u2014\u200astart it up again.<\/p>\n\n\n\n
$ .\/runserver.py &<\/pre>\n\n\n\n
Now head back to the browser and refresh the page. Links for each of your clients should immediately appear, allowing us to choose between views.<\/p>\n\n\n\n
\"\"\/
A multi-host configuration. Note links to three hosts in the left panel<\/figcaption><\/figure>\n\n\n\n
You might possibly require a server reboot and some luck to make this work the first time. It can be a bit finicky. But you\u2019ll definitely appreciate the results.<\/p>\n\n\n\n
Interpreting and acting on the results you get is your department, but my<\/em>Pluralsight Linux Performance Monitoring and Tuning course<\/em><\/a> touches on a lot of what you\u2019ll need to know. Interested in Linux administration in general? There\u2019s lots more where that came from at my Bootstrap IT site, including links to my Manning book, Linux in Action, and a hybrid course called <\/em>Linux in Motion<\/em><\/a> that\u2019s made up of more than two hours of video and around 40% of the text of Linux in Action.<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"
This article is based on my Linux Performance Monitoring and Tuning course on Pluralsight which, by coincidence, was released just before news of Spectre and Meltdown broke. By now the critical stage of the Spectre and Meltdown vulnerabilities is largely over. Patches for affected… Continue Reading →<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":349,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-348","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized"],"yoast_head":"\nCleaning up after Spectre and Meltdown: figuring out how badly they slowed down your servers - Bootstrap IT<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/bootstrap-it.com\/blog\/?p=348\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Cleaning up after Spectre and Meltdown: figuring out how badly they slowed down your servers - Bootstrap IT\" \/>\n<meta property=\"og:description\" content=\"This article is based on my Linux Performance Monitoring and Tuning course on Pluralsight which, by coincidence, was released just before news of Spectre and Meltdown broke. By now the critical stage of the Spectre and Meltdown vulnerabilities is largely over. Patches for affected… Continue Reading →\" \/>\n<meta property=\"og:url\" content=\"https:\/\/bootstrap-it.com\/blog\/?p=348\" \/>\n<meta property=\"og:site_name\" content=\"Bootstrap IT\" \/>\n<meta property=\"article:published_time\" content=\"2019-04-12T21:10:02+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/bootstrap-it.com\/blog\/wp-content\/uploads\/spectre-meltdown-linux.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1278\" \/>\n\t<meta property=\"og:image:height\" content=\"699\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"dbclin\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@davidbclinton\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"dbclin\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"9 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/bootstrap-it.com\/blog\/?p=348\",\"url\":\"https:\/\/bootstrap-it.com\/blog\/?p=348\",\"name\":\"Cleaning up after Spectre and Meltdown: figuring out how badly they slowed down your servers - Bootstrap IT\",\"isPartOf\":{\"@id\":\"https:\/\/bootstrap-it.com\/blog\/#website\"},\"datePublished\":\"2019-04-12T21:10:02+00:00\",\"dateModified\":\"2019-04-12T21:10:02+00:00\",\"author\":{\"@id\":\"https:\/\/bootstrap-it.com\/blog\/#\/schema\/person\/ae0fb1d5b3b01558b92b6426d77766ec\"},\"breadcrumb\":{\"@id\":\"https:\/\/bootstrap-it.com\/blog\/?p=348#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/bootstrap-it.com\/blog\/?p=348\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/bootstrap-it.com\/blog\/?p=348#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/bootstrap-it.com\/blog\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Cleaning up after Spectre and Meltdown: figuring out how badly they slowed down your servers\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/bootstrap-it.com\/blog\/#website\",\"url\":\"https:\/\/bootstrap-it.com\/blog\/\",\"name\":\"Bootstrap IT\",\"description\":\"Learn technology using technology\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/bootstrap-it.com\/blog\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/bootstrap-it.com\/blog\/#\/schema\/person\/ae0fb1d5b3b01558b92b6426d77766ec\",\"name\":\"dbclin\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/bootstrap-it.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/a93785d437350478a7f1dfcbec58d26bc28e0124e405179acbe1b4325c09f90a?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/a93785d437350478a7f1dfcbec58d26bc28e0124e405179acbe1b4325c09f90a?s=96&d=mm&r=g\",\"caption\":\"dbclin\"},\"sameAs\":[\"http:\/\/bootstrap-it.com\/\",\"dbclinton\",\"https:\/\/twitter.com\/davidbclinton\"],\"url\":\"https:\/\/bootstrap-it.com\/blog\/?author=1\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Cleaning up after Spectre and Meltdown: figuring out how badly they slowed down your servers - Bootstrap IT","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/bootstrap-it.com\/blog\/?p=348","og_locale":"en_US","og_type":"article","og_title":"Cleaning up after Spectre and Meltdown: figuring out how badly they slowed down your servers - Bootstrap IT","og_description":"This article is based on my Linux Performance Monitoring and Tuning course on Pluralsight which, by coincidence, was released just before news of Spectre and Meltdown broke. By now the critical stage of the Spectre and Meltdown vulnerabilities is largely over. Patches for affected… Continue Reading →","og_url":"https:\/\/bootstrap-it.com\/blog\/?p=348","og_site_name":"Bootstrap IT","article_published_time":"2019-04-12T21:10:02+00:00","og_image":[{"width":1278,"height":699,"url":"https:\/\/bootstrap-it.com\/blog\/wp-content\/uploads\/spectre-meltdown-linux.png","type":"image\/png"}],"author":"dbclin","twitter_card":"summary_large_image","twitter_creator":"@davidbclinton","twitter_misc":{"Written by":"dbclin","Est. reading time":"9 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/bootstrap-it.com\/blog\/?p=348","url":"https:\/\/bootstrap-it.com\/blog\/?p=348","name":"Cleaning up after Spectre and Meltdown: figuring out how badly they slowed down your servers - Bootstrap IT","isPartOf":{"@id":"https:\/\/bootstrap-it.com\/blog\/#website"},"datePublished":"2019-04-12T21:10:02+00:00","dateModified":"2019-04-12T21:10:02+00:00","author":{"@id":"https:\/\/bootstrap-it.com\/blog\/#\/schema\/person\/ae0fb1d5b3b01558b92b6426d77766ec"},"breadcrumb":{"@id":"https:\/\/bootstrap-it.com\/blog\/?p=348#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/bootstrap-it.com\/blog\/?p=348"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/bootstrap-it.com\/blog\/?p=348#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/bootstrap-it.com\/blog"},{"@type":"ListItem","position":2,"name":"Cleaning up after Spectre and Meltdown: figuring out how badly they slowed down your servers"}]},{"@type":"WebSite","@id":"https:\/\/bootstrap-it.com\/blog\/#website","url":"https:\/\/bootstrap-it.com\/blog\/","name":"Bootstrap IT","description":"Learn technology using technology","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/bootstrap-it.com\/blog\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/bootstrap-it.com\/blog\/#\/schema\/person\/ae0fb1d5b3b01558b92b6426d77766ec","name":"dbclin","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/bootstrap-it.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/a93785d437350478a7f1dfcbec58d26bc28e0124e405179acbe1b4325c09f90a?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/a93785d437350478a7f1dfcbec58d26bc28e0124e405179acbe1b4325c09f90a?s=96&d=mm&r=g","caption":"dbclin"},"sameAs":["http:\/\/bootstrap-it.com\/","dbclinton","https:\/\/twitter.com\/davidbclinton"],"url":"https:\/\/bootstrap-it.com\/blog\/?author=1"}]}},"_links":{"self":[{"href":"https:\/\/bootstrap-it.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/348","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/bootstrap-it.com\/blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/bootstrap-it.com\/blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/bootstrap-it.com\/blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/bootstrap-it.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=348"}],"version-history":[{"count":1,"href":"https:\/\/bootstrap-it.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/348\/revisions"}],"predecessor-version":[{"id":350,"href":"https:\/\/bootstrap-it.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/348\/revisions\/350"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/bootstrap-it.com\/blog\/index.php?rest_route=\/wp\/v2\/media\/349"}],"wp:attachment":[{"href":"https:\/\/bootstrap-it.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=348"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/bootstrap-it.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=348"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/bootstrap-it.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=348"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}