Access permissions for google-compute-engine script
I would like to run a python script on compute engine, which uses a Linux-based operating system ("debian-9-stretch-v20180307"). I am using a bash boot script with code as follows:
#! /bin/bash
sudo apt-get update
sudo apt-get -yq install python-pip
sudo pip install --upgrade google-cloud
sudo pip install --upgrade google-cloud-storage
sudo pip install --upgrade google-api-python-client
sudo pip install --upgrade google-auth-httplib2
echo "Making directories..."
mkdir -p /home/<myusername>/code
mkdir -p /home/<myusername>/rawdata
mkdir -p /home/<myusername>/etc...
sudo chown -R <myusername> /home/<myusername>
echo "Directory creation complete..."
gsutil cp gs://<mybucketname>/worker-python-code/* /home/<myusername>/code/
gsutil -m cp gs://<mybucketname>/* /home/<myusername>/rawdata/
echo "Boot complete."
When I create an instance and it boots using the script, all directories are created as intended and all data is copied per the gsutil commands. I then "SSH" into the instance and the following command to run my script:
python '/home/<myusername>/code/workermaster.py'
The script runs, until it gets to a line which requires it to create .csv on the disk, and then I get an "[Errono 13] Permission denied" error.
I have done a few tests and attempts to get the script running when run an SSH shell based on other threads and google searches but cannot seem to get it working. Some interesting notes and observations are:
1) In the "console output" reports there is a line which reads:
chown: invalid user: ‘<myusername>’
And then many lines down it states:
Created user account <myusername>
Which suggests I am trying to set ownership to someone who doesnt exist yet, but I cant work out what to do about it.
2) If I run the "chown" command line after logging in via SSH and then run my script, it runs exactly as intended with no issues.
3) I tried to fix the issue by adding a line to the boot script which read:
sudo chmod -R 777 /home/
When I used this command, I couldnt even SSH into the instance.
Ultimately I plan to add "nohang up" line to the script so that the python code runs remotely without me logging in, but I need to understand what is happening here and to be able to just SSH in and run my script following boot.
ssh chown google-cloud
New contributor
add a comment |
I would like to run a python script on compute engine, which uses a Linux-based operating system ("debian-9-stretch-v20180307"). I am using a bash boot script with code as follows:
#! /bin/bash
sudo apt-get update
sudo apt-get -yq install python-pip
sudo pip install --upgrade google-cloud
sudo pip install --upgrade google-cloud-storage
sudo pip install --upgrade google-api-python-client
sudo pip install --upgrade google-auth-httplib2
echo "Making directories..."
mkdir -p /home/<myusername>/code
mkdir -p /home/<myusername>/rawdata
mkdir -p /home/<myusername>/etc...
sudo chown -R <myusername> /home/<myusername>
echo "Directory creation complete..."
gsutil cp gs://<mybucketname>/worker-python-code/* /home/<myusername>/code/
gsutil -m cp gs://<mybucketname>/* /home/<myusername>/rawdata/
echo "Boot complete."
When I create an instance and it boots using the script, all directories are created as intended and all data is copied per the gsutil commands. I then "SSH" into the instance and the following command to run my script:
python '/home/<myusername>/code/workermaster.py'
The script runs, until it gets to a line which requires it to create .csv on the disk, and then I get an "[Errono 13] Permission denied" error.
I have done a few tests and attempts to get the script running when run an SSH shell based on other threads and google searches but cannot seem to get it working. Some interesting notes and observations are:
1) In the "console output" reports there is a line which reads:
chown: invalid user: ‘<myusername>’
And then many lines down it states:
Created user account <myusername>
Which suggests I am trying to set ownership to someone who doesnt exist yet, but I cant work out what to do about it.
2) If I run the "chown" command line after logging in via SSH and then run my script, it runs exactly as intended with no issues.
3) I tried to fix the issue by adding a line to the boot script which read:
sudo chmod -R 777 /home/
When I used this command, I couldnt even SSH into the instance.
Ultimately I plan to add "nohang up" line to the script so that the python code runs remotely without me logging in, but I need to understand what is happening here and to be able to just SSH in and run my script following boot.
ssh chown google-cloud
New contributor
add a comment |
I would like to run a python script on compute engine, which uses a Linux-based operating system ("debian-9-stretch-v20180307"). I am using a bash boot script with code as follows:
#! /bin/bash
sudo apt-get update
sudo apt-get -yq install python-pip
sudo pip install --upgrade google-cloud
sudo pip install --upgrade google-cloud-storage
sudo pip install --upgrade google-api-python-client
sudo pip install --upgrade google-auth-httplib2
echo "Making directories..."
mkdir -p /home/<myusername>/code
mkdir -p /home/<myusername>/rawdata
mkdir -p /home/<myusername>/etc...
sudo chown -R <myusername> /home/<myusername>
echo "Directory creation complete..."
gsutil cp gs://<mybucketname>/worker-python-code/* /home/<myusername>/code/
gsutil -m cp gs://<mybucketname>/* /home/<myusername>/rawdata/
echo "Boot complete."
When I create an instance and it boots using the script, all directories are created as intended and all data is copied per the gsutil commands. I then "SSH" into the instance and the following command to run my script:
python '/home/<myusername>/code/workermaster.py'
The script runs, until it gets to a line which requires it to create .csv on the disk, and then I get an "[Errono 13] Permission denied" error.
I have done a few tests and attempts to get the script running when run an SSH shell based on other threads and google searches but cannot seem to get it working. Some interesting notes and observations are:
1) In the "console output" reports there is a line which reads:
chown: invalid user: ‘<myusername>’
And then many lines down it states:
Created user account <myusername>
Which suggests I am trying to set ownership to someone who doesnt exist yet, but I cant work out what to do about it.
2) If I run the "chown" command line after logging in via SSH and then run my script, it runs exactly as intended with no issues.
3) I tried to fix the issue by adding a line to the boot script which read:
sudo chmod -R 777 /home/
When I used this command, I couldnt even SSH into the instance.
Ultimately I plan to add "nohang up" line to the script so that the python code runs remotely without me logging in, but I need to understand what is happening here and to be able to just SSH in and run my script following boot.
ssh chown google-cloud
New contributor
I would like to run a python script on compute engine, which uses a Linux-based operating system ("debian-9-stretch-v20180307"). I am using a bash boot script with code as follows:
#! /bin/bash
sudo apt-get update
sudo apt-get -yq install python-pip
sudo pip install --upgrade google-cloud
sudo pip install --upgrade google-cloud-storage
sudo pip install --upgrade google-api-python-client
sudo pip install --upgrade google-auth-httplib2
echo "Making directories..."
mkdir -p /home/<myusername>/code
mkdir -p /home/<myusername>/rawdata
mkdir -p /home/<myusername>/etc...
sudo chown -R <myusername> /home/<myusername>
echo "Directory creation complete..."
gsutil cp gs://<mybucketname>/worker-python-code/* /home/<myusername>/code/
gsutil -m cp gs://<mybucketname>/* /home/<myusername>/rawdata/
echo "Boot complete."
When I create an instance and it boots using the script, all directories are created as intended and all data is copied per the gsutil commands. I then "SSH" into the instance and the following command to run my script:
python '/home/<myusername>/code/workermaster.py'
The script runs, until it gets to a line which requires it to create .csv on the disk, and then I get an "[Errono 13] Permission denied" error.
I have done a few tests and attempts to get the script running when run an SSH shell based on other threads and google searches but cannot seem to get it working. Some interesting notes and observations are:
1) In the "console output" reports there is a line which reads:
chown: invalid user: ‘<myusername>’
And then many lines down it states:
Created user account <myusername>
Which suggests I am trying to set ownership to someone who doesnt exist yet, but I cant work out what to do about it.
2) If I run the "chown" command line after logging in via SSH and then run my script, it runs exactly as intended with no issues.
3) I tried to fix the issue by adding a line to the boot script which read:
sudo chmod -R 777 /home/
When I used this command, I couldnt even SSH into the instance.
Ultimately I plan to add "nohang up" line to the script so that the python code runs remotely without me logging in, but I need to understand what is happening here and to be able to just SSH in and run my script following boot.
ssh chown google-cloud
ssh chown google-cloud
New contributor
New contributor
edited 5 hours ago
Rui F Ribeiro
41.3k1481140
41.3k1481140
New contributor
asked 6 hours ago
PaulPaul
11
11
New contributor
New contributor
add a comment |
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "106"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Paul is a new contributor. Be nice, and check out our Code of Conduct.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f505384%2faccess-permissions-for-google-compute-engine-script%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Paul is a new contributor. Be nice, and check out our Code of Conduct.
Paul is a new contributor. Be nice, and check out our Code of Conduct.
Paul is a new contributor. Be nice, and check out our Code of Conduct.
Paul is a new contributor. Be nice, and check out our Code of Conduct.
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f505384%2faccess-permissions-for-google-compute-engine-script%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown