Slicing JSON with JQ to Create Arrays for New Relic Ingestion
up vote
0
down vote
favorite
sorry for the long title.
I need to take some JSON, slice it using JQ so New Relic can parse it properly as it only takes a specific format when pushing JSON to their dashboards.
I am using a bash script and WPCLI and then formatting the JSONs output using JQ.
Below is my script.
#!/bin/bash
#for d in /var/www/* ; do
# echo "$d"
#done
for f in /var/www/*/public_html/; do
if [[ -d $f ]]; then
#echo "$f"
cd $f;
SITEURL="$(wp option get siteurl)"
USERS="$(wp user list --fields=display_name,user_email,user_registered,roles --format=json | jq .)"
PLUGINS="$(wp plugin list --format=json | jq .)"
echo "${SITEURL}" "${USERS}" "${PLUGINS}"
cd ../.. ;
fi
done
Below is the output I am getting...
cat wp-info-output
http://mytest1.com {
"display_name": "testuser1",
"user_email": "test.user@testing.com",
"user_registered": "2018-11-26 17:44:09",
"roles": "administrator"
} {
"name": "akismet",
"status": "inactive",
"update": "available",
"version": "4.0.8"
}
{
"name": "hello",
"status": "inactive",
"update": "none",
"version": "1.7"
}
I need it to look like this...
[
{
"eventType":"WordpressSite",
"siteurl":"http://mytest1.com"
},
{
"eventType":"WordpressPlugins",
"pluginName": "akismet",
"status": "inactive",
"update": "available",
"version": "4.0.8"
},
{
"eventType":"WordpressPlugins",
"pluginName": "hello",
"status": "inactive",
"update": "none",
"version": "1.7"
}
{
"eventType":"Users",
"display_name": "testuser2",
"user_email": "test.user@testing.com",
"user_registered": "2018-11-26 17:44:04",
"roles": "administrator"
}
]
Thanks in advance for any tips. I think jq is my answer for manipulating the JSON to be formatted but I am hitting some walls.
linux text-processing json jq wordpress
add a comment |
up vote
0
down vote
favorite
sorry for the long title.
I need to take some JSON, slice it using JQ so New Relic can parse it properly as it only takes a specific format when pushing JSON to their dashboards.
I am using a bash script and WPCLI and then formatting the JSONs output using JQ.
Below is my script.
#!/bin/bash
#for d in /var/www/* ; do
# echo "$d"
#done
for f in /var/www/*/public_html/; do
if [[ -d $f ]]; then
#echo "$f"
cd $f;
SITEURL="$(wp option get siteurl)"
USERS="$(wp user list --fields=display_name,user_email,user_registered,roles --format=json | jq .)"
PLUGINS="$(wp plugin list --format=json | jq .)"
echo "${SITEURL}" "${USERS}" "${PLUGINS}"
cd ../.. ;
fi
done
Below is the output I am getting...
cat wp-info-output
http://mytest1.com {
"display_name": "testuser1",
"user_email": "test.user@testing.com",
"user_registered": "2018-11-26 17:44:09",
"roles": "administrator"
} {
"name": "akismet",
"status": "inactive",
"update": "available",
"version": "4.0.8"
}
{
"name": "hello",
"status": "inactive",
"update": "none",
"version": "1.7"
}
I need it to look like this...
[
{
"eventType":"WordpressSite",
"siteurl":"http://mytest1.com"
},
{
"eventType":"WordpressPlugins",
"pluginName": "akismet",
"status": "inactive",
"update": "available",
"version": "4.0.8"
},
{
"eventType":"WordpressPlugins",
"pluginName": "hello",
"status": "inactive",
"update": "none",
"version": "1.7"
}
{
"eventType":"Users",
"display_name": "testuser2",
"user_email": "test.user@testing.com",
"user_registered": "2018-11-26 17:44:04",
"roles": "administrator"
}
]
Thanks in advance for any tips. I think jq is my answer for manipulating the JSON to be formatted but I am hitting some walls.
linux text-processing json jq wordpress
Note that your directory variable$f
holds an absolute path, so there's really no need tocd ../..
in the loop
– glenn jackman
Nov 30 at 20:44
add a comment |
up vote
0
down vote
favorite
up vote
0
down vote
favorite
sorry for the long title.
I need to take some JSON, slice it using JQ so New Relic can parse it properly as it only takes a specific format when pushing JSON to their dashboards.
I am using a bash script and WPCLI and then formatting the JSONs output using JQ.
Below is my script.
#!/bin/bash
#for d in /var/www/* ; do
# echo "$d"
#done
for f in /var/www/*/public_html/; do
if [[ -d $f ]]; then
#echo "$f"
cd $f;
SITEURL="$(wp option get siteurl)"
USERS="$(wp user list --fields=display_name,user_email,user_registered,roles --format=json | jq .)"
PLUGINS="$(wp plugin list --format=json | jq .)"
echo "${SITEURL}" "${USERS}" "${PLUGINS}"
cd ../.. ;
fi
done
Below is the output I am getting...
cat wp-info-output
http://mytest1.com {
"display_name": "testuser1",
"user_email": "test.user@testing.com",
"user_registered": "2018-11-26 17:44:09",
"roles": "administrator"
} {
"name": "akismet",
"status": "inactive",
"update": "available",
"version": "4.0.8"
}
{
"name": "hello",
"status": "inactive",
"update": "none",
"version": "1.7"
}
I need it to look like this...
[
{
"eventType":"WordpressSite",
"siteurl":"http://mytest1.com"
},
{
"eventType":"WordpressPlugins",
"pluginName": "akismet",
"status": "inactive",
"update": "available",
"version": "4.0.8"
},
{
"eventType":"WordpressPlugins",
"pluginName": "hello",
"status": "inactive",
"update": "none",
"version": "1.7"
}
{
"eventType":"Users",
"display_name": "testuser2",
"user_email": "test.user@testing.com",
"user_registered": "2018-11-26 17:44:04",
"roles": "administrator"
}
]
Thanks in advance for any tips. I think jq is my answer for manipulating the JSON to be formatted but I am hitting some walls.
linux text-processing json jq wordpress
sorry for the long title.
I need to take some JSON, slice it using JQ so New Relic can parse it properly as it only takes a specific format when pushing JSON to their dashboards.
I am using a bash script and WPCLI and then formatting the JSONs output using JQ.
Below is my script.
#!/bin/bash
#for d in /var/www/* ; do
# echo "$d"
#done
for f in /var/www/*/public_html/; do
if [[ -d $f ]]; then
#echo "$f"
cd $f;
SITEURL="$(wp option get siteurl)"
USERS="$(wp user list --fields=display_name,user_email,user_registered,roles --format=json | jq .)"
PLUGINS="$(wp plugin list --format=json | jq .)"
echo "${SITEURL}" "${USERS}" "${PLUGINS}"
cd ../.. ;
fi
done
Below is the output I am getting...
cat wp-info-output
http://mytest1.com {
"display_name": "testuser1",
"user_email": "test.user@testing.com",
"user_registered": "2018-11-26 17:44:09",
"roles": "administrator"
} {
"name": "akismet",
"status": "inactive",
"update": "available",
"version": "4.0.8"
}
{
"name": "hello",
"status": "inactive",
"update": "none",
"version": "1.7"
}
I need it to look like this...
[
{
"eventType":"WordpressSite",
"siteurl":"http://mytest1.com"
},
{
"eventType":"WordpressPlugins",
"pluginName": "akismet",
"status": "inactive",
"update": "available",
"version": "4.0.8"
},
{
"eventType":"WordpressPlugins",
"pluginName": "hello",
"status": "inactive",
"update": "none",
"version": "1.7"
}
{
"eventType":"Users",
"display_name": "testuser2",
"user_email": "test.user@testing.com",
"user_registered": "2018-11-26 17:44:04",
"roles": "administrator"
}
]
Thanks in advance for any tips. I think jq is my answer for manipulating the JSON to be formatted but I am hitting some walls.
linux text-processing json jq wordpress
linux text-processing json jq wordpress
edited Nov 30 at 17:49
Jeff Schaller
37k1052121
37k1052121
asked Nov 30 at 17:48
Jovan Hernandez
133
133
Note that your directory variable$f
holds an absolute path, so there's really no need tocd ../..
in the loop
– glenn jackman
Nov 30 at 20:44
add a comment |
Note that your directory variable$f
holds an absolute path, so there's really no need tocd ../..
in the loop
– glenn jackman
Nov 30 at 20:44
Note that your directory variable
$f
holds an absolute path, so there's really no need to cd ../..
in the loop– glenn jackman
Nov 30 at 20:44
Note that your directory variable
$f
holds an absolute path, so there's really no need to cd ../..
in the loop– glenn jackman
Nov 30 at 20:44
add a comment |
1 Answer
1
active
oldest
votes
up vote
1
down vote
It seems you want to take all those JSON objects and smush them together in one bit array: try
siteurl=$(wp option get siteurl)
users=$(
wp user list --fields=display_name,user_email,user_registered,roles --format=json) |
jq '. | .eventType = "WordpressUser"'
)
plugins=$( wp plugin list --format=json | jq '. | .eventType = "WordpressPlugin"' )
{
printf '{"eventType":"WordpressSite","siteurl":"%s"}n' "$(wp option get siteurl)"
echo "$plugins"
echo "$users"
} | jq -s .
Yea I know it seems goofy but that's how New Relic want's their json events formatted. Each JSON event should be it's own object, without any real arrays. Therefore, each event would need to be its own object with a header event type. You can see more here: docs.newrelic.com/docs/insights/insights-data-sources/… But I pasted the example New Relic wants.
– Jovan Hernandez
Nov 30 at 21:27
Also, what you provided got me 99% there! The only thing is the eventType for the plugins is at the bottom of the JSON object instead of at the top.
– Jovan Hernandez
Nov 30 at 21:41
The order should not matter. The attributes of a JSON object are not inherently ordered.
– glenn jackman
Nov 30 at 21:42
Ah you're right, it really shouldn't matter. For some reason the Users are not being outputted but I think you got me 99% there and I can continue to hack it out. Thanks!
– Jovan Hernandez
Nov 30 at 21:46
add a comment |
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
1
down vote
It seems you want to take all those JSON objects and smush them together in one bit array: try
siteurl=$(wp option get siteurl)
users=$(
wp user list --fields=display_name,user_email,user_registered,roles --format=json) |
jq '. | .eventType = "WordpressUser"'
)
plugins=$( wp plugin list --format=json | jq '. | .eventType = "WordpressPlugin"' )
{
printf '{"eventType":"WordpressSite","siteurl":"%s"}n' "$(wp option get siteurl)"
echo "$plugins"
echo "$users"
} | jq -s .
Yea I know it seems goofy but that's how New Relic want's their json events formatted. Each JSON event should be it's own object, without any real arrays. Therefore, each event would need to be its own object with a header event type. You can see more here: docs.newrelic.com/docs/insights/insights-data-sources/… But I pasted the example New Relic wants.
– Jovan Hernandez
Nov 30 at 21:27
Also, what you provided got me 99% there! The only thing is the eventType for the plugins is at the bottom of the JSON object instead of at the top.
– Jovan Hernandez
Nov 30 at 21:41
The order should not matter. The attributes of a JSON object are not inherently ordered.
– glenn jackman
Nov 30 at 21:42
Ah you're right, it really shouldn't matter. For some reason the Users are not being outputted but I think you got me 99% there and I can continue to hack it out. Thanks!
– Jovan Hernandez
Nov 30 at 21:46
add a comment |
up vote
1
down vote
It seems you want to take all those JSON objects and smush them together in one bit array: try
siteurl=$(wp option get siteurl)
users=$(
wp user list --fields=display_name,user_email,user_registered,roles --format=json) |
jq '. | .eventType = "WordpressUser"'
)
plugins=$( wp plugin list --format=json | jq '. | .eventType = "WordpressPlugin"' )
{
printf '{"eventType":"WordpressSite","siteurl":"%s"}n' "$(wp option get siteurl)"
echo "$plugins"
echo "$users"
} | jq -s .
Yea I know it seems goofy but that's how New Relic want's their json events formatted. Each JSON event should be it's own object, without any real arrays. Therefore, each event would need to be its own object with a header event type. You can see more here: docs.newrelic.com/docs/insights/insights-data-sources/… But I pasted the example New Relic wants.
– Jovan Hernandez
Nov 30 at 21:27
Also, what you provided got me 99% there! The only thing is the eventType for the plugins is at the bottom of the JSON object instead of at the top.
– Jovan Hernandez
Nov 30 at 21:41
The order should not matter. The attributes of a JSON object are not inherently ordered.
– glenn jackman
Nov 30 at 21:42
Ah you're right, it really shouldn't matter. For some reason the Users are not being outputted but I think you got me 99% there and I can continue to hack it out. Thanks!
– Jovan Hernandez
Nov 30 at 21:46
add a comment |
up vote
1
down vote
up vote
1
down vote
It seems you want to take all those JSON objects and smush them together in one bit array: try
siteurl=$(wp option get siteurl)
users=$(
wp user list --fields=display_name,user_email,user_registered,roles --format=json) |
jq '. | .eventType = "WordpressUser"'
)
plugins=$( wp plugin list --format=json | jq '. | .eventType = "WordpressPlugin"' )
{
printf '{"eventType":"WordpressSite","siteurl":"%s"}n' "$(wp option get siteurl)"
echo "$plugins"
echo "$users"
} | jq -s .
It seems you want to take all those JSON objects and smush them together in one bit array: try
siteurl=$(wp option get siteurl)
users=$(
wp user list --fields=display_name,user_email,user_registered,roles --format=json) |
jq '. | .eventType = "WordpressUser"'
)
plugins=$( wp plugin list --format=json | jq '. | .eventType = "WordpressPlugin"' )
{
printf '{"eventType":"WordpressSite","siteurl":"%s"}n' "$(wp option get siteurl)"
echo "$plugins"
echo "$users"
} | jq -s .
edited Nov 30 at 21:43
answered Nov 30 at 20:42
glenn jackman
49.6k569106
49.6k569106
Yea I know it seems goofy but that's how New Relic want's their json events formatted. Each JSON event should be it's own object, without any real arrays. Therefore, each event would need to be its own object with a header event type. You can see more here: docs.newrelic.com/docs/insights/insights-data-sources/… But I pasted the example New Relic wants.
– Jovan Hernandez
Nov 30 at 21:27
Also, what you provided got me 99% there! The only thing is the eventType for the plugins is at the bottom of the JSON object instead of at the top.
– Jovan Hernandez
Nov 30 at 21:41
The order should not matter. The attributes of a JSON object are not inherently ordered.
– glenn jackman
Nov 30 at 21:42
Ah you're right, it really shouldn't matter. For some reason the Users are not being outputted but I think you got me 99% there and I can continue to hack it out. Thanks!
– Jovan Hernandez
Nov 30 at 21:46
add a comment |
Yea I know it seems goofy but that's how New Relic want's their json events formatted. Each JSON event should be it's own object, without any real arrays. Therefore, each event would need to be its own object with a header event type. You can see more here: docs.newrelic.com/docs/insights/insights-data-sources/… But I pasted the example New Relic wants.
– Jovan Hernandez
Nov 30 at 21:27
Also, what you provided got me 99% there! The only thing is the eventType for the plugins is at the bottom of the JSON object instead of at the top.
– Jovan Hernandez
Nov 30 at 21:41
The order should not matter. The attributes of a JSON object are not inherently ordered.
– glenn jackman
Nov 30 at 21:42
Ah you're right, it really shouldn't matter. For some reason the Users are not being outputted but I think you got me 99% there and I can continue to hack it out. Thanks!
– Jovan Hernandez
Nov 30 at 21:46
Yea I know it seems goofy but that's how New Relic want's their json events formatted. Each JSON event should be it's own object, without any real arrays. Therefore, each event would need to be its own object with a header event type. You can see more here: docs.newrelic.com/docs/insights/insights-data-sources/… But I pasted the example New Relic wants.
– Jovan Hernandez
Nov 30 at 21:27
Yea I know it seems goofy but that's how New Relic want's their json events formatted. Each JSON event should be it's own object, without any real arrays. Therefore, each event would need to be its own object with a header event type. You can see more here: docs.newrelic.com/docs/insights/insights-data-sources/… But I pasted the example New Relic wants.
– Jovan Hernandez
Nov 30 at 21:27
Also, what you provided got me 99% there! The only thing is the eventType for the plugins is at the bottom of the JSON object instead of at the top.
– Jovan Hernandez
Nov 30 at 21:41
Also, what you provided got me 99% there! The only thing is the eventType for the plugins is at the bottom of the JSON object instead of at the top.
– Jovan Hernandez
Nov 30 at 21:41
The order should not matter. The attributes of a JSON object are not inherently ordered.
– glenn jackman
Nov 30 at 21:42
The order should not matter. The attributes of a JSON object are not inherently ordered.
– glenn jackman
Nov 30 at 21:42
Ah you're right, it really shouldn't matter. For some reason the Users are not being outputted but I think you got me 99% there and I can continue to hack it out. Thanks!
– Jovan Hernandez
Nov 30 at 21:46
Ah you're right, it really shouldn't matter. For some reason the Users are not being outputted but I think you got me 99% there and I can continue to hack it out. Thanks!
– Jovan Hernandez
Nov 30 at 21:46
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f485212%2fslicing-json-with-jq-to-create-arrays-for-new-relic-ingestion%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Note that your directory variable
$f
holds an absolute path, so there's really no need tocd ../..
in the loop– glenn jackman
Nov 30 at 20:44