所以,我在这里:我正在用 Python 编写一个脚本,它执行以下操作:
我将在最后附上这两个脚本,但我怀疑它们对那个问题有任何用处。
为了不等待通过 SSH 启动的日志记录脚本结束以继续执行 python 脚本,我使用 -f 和 SSH 在后台发送它。
python 脚本确实继续,但我发现自己遇到了与日志记录过程和基准测试的时间相关的问题,这可能会因客户端的网络访问和处理能力而有很大差异。
我希望以允许自己继续使用客户端上的 python 脚本执行基准测试的方式在服务器上启动日志记录命令,但我也不想在日志记录完成服务器之前启动绘图客户端边。现在我不能通过计算近似 sleep 来做到这一点,因为它会因客户端和连接而异。
因此,我想仅在客户端上退出日志记录脚本时才开始检索日志记录文件并进行绘图,这将是 SSH 命令的成功执行和退出。
如何在 python 中跟踪 SSH 命令状态,以便仅在 SSH 命令(在后台)退出时启动脚本的那一部分?
我想避免使用线程,因为它不太适合脚本。我当然愿意接受任何我没有想到的更好的建议。
日志文件,服务器端
#!/usr/bin/env bash
#perf_measurement.sh
# logs the load average every n seconds for a y period of time in a csv file
# interval, in seconds, for the load measure
INTERVAL=1
# amount of measures
MEASURES=$1
# path of the logging file
LOGFILE_EXT=".csv"
LOGFILE="$2""$LOGFILE_EXT"
# separator for the csv file
SEPARATOR=","
# load average file
LOADFILE="/proc/loadavg"
# timestamp at start of measure/script
INIT_TIMESTAMP=`date +%s`
# clearing logfile
if [ -e $LOGFILE ]; then
rm $LOGFILE
fi
# function to get one of the load average values from LOADFILE
loadpm () {
cat $LOADFILE | cut -d' ' -f$1
}
# function generation a timestamp starting from the start of the script, in seconds
timestamp () {
echo $(expr $(date +%s) - $INIT_TIMESTAMP)
}
for (( i=0; i<$MEASURES; i++ )) ; do
echo $(timestamp)$SEPARATOR$(loadpm 1) >> $LOGFILE
sleep $INTERVAL
done
exit 0
Python 脚本,客户端
#!/usr/bin/python
#benchmark.py
import csv
import matplotlib.pyplot as plt
import os
import time
SERVER = "edited"
SERVER_USER = "ubuntu"
SERVER_PERF_MEASUREMENT_SCRIPT_PATH = "/var/www/html"
SERVER_PERF_MEASUREMENT_SCRIPT = "perf_measurement.sh"
REQUESTS = 1000000 # high because time limitation is used instead of number of requests to facilitate logging
CONCURRENCY = input("Input concurrency for test : ")
MEASURE_TIME = input("Input time of the test (in seconds) : ")
FILE = input("Input name of file/repo for test (without the extension) : ")
TEST_NAME = input("Input a name for the test (no space) : ")
FILENAME= f"{TEST_NAME}_conc_{CONCURRENCY}"
GRAPH_FILE = f"{FILENAME}.png"
LOG_FILE = f"{FILENAME}.csv"
FILE_PATH = f"/{FILE}"
TEST_TIME = int(int(MEASURE_TIME) - int(MEASURE_TIME) / 4)
SLEEP_TIME = int(int(MEASURE_TIME) / 8)
SSH_COMMAND = f"ssh -f {SERVER_USER}@{SERVER} 'cd {SERVER_PERF_MEASUREMENT_SCRIPT_PATH}&&./"\
f"{SERVER_PERF_MEASUREMENT_SCRIPT} {MEASURE_TIME} {FILENAME}'"
AB_COMMAND = f"ab -c {CONCURRENCY} -t {TEST_TIME} -n {REQUESTS} {SERVER}{FILE_PATH}"
SCP_COMMAND = f"scp {SERVER_USER}@{SERVER}:{SERVER_PERF_MEASUREMENT_SCRIPT_PATH}/{LOG_FILE} ."
print(SSH_COMMAND)
print("\nStarting the logging server side...\n")
os.system(SSH_COMMAND)
print(f"\nSleeping {SLEEP_TIME} seconds to visualize load gain...\n")
time.sleep(SLEEP_TIME)
print("\nStarting benchmark...\n")
os.system(AB_COMMAND)
print("\nRecovering log file from server...\n")
os.system(SCP_COMMAND)
print("\nGenerating graph...\n")
# declaring list for coordinates of the graph
x_coord = []
y_coord = []
# opening log file
with open(LOG_FILE, "r") as csvfile:
# reading csv logfile
logfile = csv.reader(csvfile)
# iterating through logfile
for row in logfile:
# storing coordinates in list
x_coord.append(int(row[0]))
y_coord.append(float(row[1]))
# generation graph
plt.plot(x_coord, y_coord)
plt.ylabel('Load average (from /proc/loadavg)')
plt.xlabel('Time')
# exporting graph to png
plt.savefig(GRAPH_FILE)
请您参考如下方法:
鉴于此脚本相当独特的性质,它似乎是 subprocess.Popen(...) 的一个很好的用例。 .
与其将 SSH 命令发送到后台,不如使用 Popen 通过进程句柄来启动它。然后,您可以继续执行您需要在您身边执行的任何测试并返回到您之前的那个进程句柄 scp结果。这应该有效,因为 ssh command,当在其方法调用结束时提供一组命令时,发送命令而不是打开交互式登录 shell,然后将输出返回到客户端计算机。远程命令完成后,ssh关闭连接并退出自己的进程。因此,以下应该起作用:
import subprocess
import shlex
# ... your other imports and code...
_PROC_TIMEOUT_SEC = 5
SSH_COMMAND = (
f"ssh {SERVER_USER}@{SERVER} 'cd {SERVER_PERF_MEASUREMENT_SCRIPT_PATH}&&./"
f"{SERVER_PERF_MEASUREMENT_SCRIPT} {MEASURE_TIME} {FILENAME}'"
)
AB_COMMAND = f"ab -c {CONCURRENCY} -t {TEST_TIME} -n {REQUESTS} {SERVER}{FILE_PATH}"
SCP_COMMAND = f"scp {SERVER_USER}@{SERVER}:{SERVER_PERF_MEASUREMENT_SCRIPT_PATH}/{LOG_FILE} ."
print("\nStarting the logging server side...\n")
portioned_ssh_command = shlex.split(SSH_COMMAND)
ssh_proc = subprocess.Popen(portioned_ssh_command)
# Still perform your testing as you had planned...
# Not sure if you need to sleep here, but I'll leave that up to you.
print(f"\nSleeping {SLEEP_TIME} seconds to visualize load gain...\n")
time.sleep(SLEEP_TIME)
print("\nStarting benchmark...\n")
os.system(AB_COMMAND)
# Wait for SSH process to finish before attempting to scp log file.
ssh_return_code = None
while(ssh_return_code is None):
try:
ssh_return_code = ssh_proc.wait(_PROC_TIMEOUT_SEC)
except subprocess.TimeoutExpired:
pass
print(f"Info: SSH completed with exitcode {ssh_proc.returncode}")
# Now that SSH
print("\nRecovering log file from server...\n")
os.system(SCP_COMMAND)
# ... The rest of your code...
备注 :我没有测试上面的代码,但我非常有信心它会做你想要完成的事情。




