所以,我在这里:我正在用 Python 编写一个脚本,它执行以下操作:

  • 启动命令以通过 SSH 在服务器上启动日志记录进程(bash 脚本)
  • 使用 ab 从客户端对服务器进行基准测试
  • 使用 scp
  • 恢复服务器上生成的日志文件
  • 使用 matplotlib 生成带有客户端数据的图形

  • 我将在最后附上这两个脚本,但我怀疑它们对那个问题有任何用处。

    为了不等待通过 SSH 启动的日志记录脚本结束以继续执行 python 脚本,我使用 -f 和 SSH 在后台发送它。
    python 脚本确实继续,但我发现自己遇到了与日志记录过程和基准测试的时间相关的问题,这可能会因客户端的网络访问和处理能力而有很大差异。

    我希望以允许自己继续使用客户端上的 python 脚本执行基准测试的方式在服务器上启动日志记录命令,但我也不想在日志记录完成服务器之前启动绘图客户端边。现在我不能通过计算近似 sleep 来做到这一点,因为它会因客户端和连接而异。

    因此,我想仅在客户端上退出日志记录脚本时才开始检索日志记录文件并进行绘图,这将是 SSH 命令的成功执行和退出。

    如何在 python 中跟踪 SSH 命令状态,以便仅在 SSH 命令(在后台)退出时启动脚本的那一部分?

    我想避免使用线程,因为它不太适合脚本。我当然愿意接受任何我没有想到的更好的建议。

    日志文件,服务器端
    #!/usr/bin/env bash 
    #perf_measurement.sh 
     
    # logs the load average every n seconds for a y period of time in a csv file 
     
    # interval, in seconds, for the load measure 
    INTERVAL=1 
    # amount of measures 
    MEASURES=$1 
    # path of the logging file 
    LOGFILE_EXT=".csv" 
    LOGFILE="$2""$LOGFILE_EXT" 
    # separator for the csv file 
    SEPARATOR="," 
    # load average file 
    LOADFILE="/proc/loadavg" 
    # timestamp at start of measure/script 
    INIT_TIMESTAMP=`date +%s` 
     
    # clearing logfile 
    if [ -e $LOGFILE ]; then 
      rm $LOGFILE 
    fi 
     
    # function to get one of the load average values from LOADFILE 
    loadpm () { 
      cat $LOADFILE | cut -d' ' -f$1 
    } 
    # function generation a timestamp starting from the start of the script, in seconds 
    timestamp () { 
      echo $(expr $(date +%s) - $INIT_TIMESTAMP) 
    } 
     
    for (( i=0; i<$MEASURES; i++ )) ; do 
      echo $(timestamp)$SEPARATOR$(loadpm 1) >> $LOGFILE 
      sleep $INTERVAL 
    done 
     
     
    exit 0 
    

    Python 脚本,客户端
    #!/usr/bin/python
    #benchmark.py
    import csv 
    import matplotlib.pyplot as plt 
    import os 
    import time 
     
    SERVER = "edited" 
    SERVER_USER = "ubuntu" 
    SERVER_PERF_MEASUREMENT_SCRIPT_PATH = "/var/www/html" 
    SERVER_PERF_MEASUREMENT_SCRIPT = "perf_measurement.sh" 
    REQUESTS = 1000000 # high because time limitation is used instead of number of requests to facilitate logging 
     
    CONCURRENCY = input("Input concurrency for test : ") 
    MEASURE_TIME = input("Input time of the test (in seconds) : ") 
    FILE = input("Input name of file/repo for test (without the extension) : ") 
    TEST_NAME = input("Input a name for the test (no space) : ") 
    FILENAME= f"{TEST_NAME}_conc_{CONCURRENCY}" 
    GRAPH_FILE = f"{FILENAME}.png" 
    LOG_FILE = f"{FILENAME}.csv" 
     
    FILE_PATH = f"/{FILE}" 
    TEST_TIME = int(int(MEASURE_TIME) - int(MEASURE_TIME) / 4) 
    SLEEP_TIME = int(int(MEASURE_TIME) / 8) 
     
     
    SSH_COMMAND = f"ssh -f {SERVER_USER}@{SERVER} 'cd {SERVER_PERF_MEASUREMENT_SCRIPT_PATH}&&./"\ 
                    f"{SERVER_PERF_MEASUREMENT_SCRIPT} {MEASURE_TIME} {FILENAME}'" 
    AB_COMMAND = f"ab -c {CONCURRENCY} -t {TEST_TIME} -n {REQUESTS} {SERVER}{FILE_PATH}" 
    SCP_COMMAND = f"scp {SERVER_USER}@{SERVER}:{SERVER_PERF_MEASUREMENT_SCRIPT_PATH}/{LOG_FILE} ." 
     
    print(SSH_COMMAND) 
     
    print("\nStarting the logging server side...\n") 
    os.system(SSH_COMMAND) 
     
    print(f"\nSleeping {SLEEP_TIME} seconds to visualize load gain...\n") 
    time.sleep(SLEEP_TIME) 
     
    print("\nStarting benchmark...\n") 
    os.system(AB_COMMAND) 
     
    print("\nRecovering log file from server...\n") 
    os.system(SCP_COMMAND) 
     
     
    print("\nGenerating graph...\n") 
    # declaring list for coordinates of the graph 
    x_coord = [] 
    y_coord = [] 
     
    # opening log file 
    with open(LOG_FILE, "r") as csvfile: 
     
        # reading csv logfile 
        logfile = csv.reader(csvfile) 
     
        # iterating through logfile 
        for row in logfile: 
            # storing coordinates in list 
            x_coord.append(int(row[0])) 
            y_coord.append(float(row[1])) 
     
     
    # generation graph 
    plt.plot(x_coord, y_coord) 
    plt.ylabel('Load average (from /proc/loadavg)') 
    plt.xlabel('Time') 
     
    # exporting graph to png 
    plt.savefig(GRAPH_FILE) 
    

    请您参考如下方法:

    鉴于此脚本相当独特的性质,它似乎是 subprocess.Popen(...) 的一个很好的用例。 .

    与其将 SSH 命令发送到后台,不如使用 Popen 通过进程句柄来启动它。然后,您可以继续执行您需要在您身边执行的任何测试并返回到您之前的那个进程句柄 scp结果。这应该有效,因为 ssh command,当在其方法调用结束时提供一组命令时,发送命令而不是打开交互式登录 shell,然后将输出返回到客户端计算机。远程命令完成后,ssh关闭连接并退出自己的进程。因此,以下应该起作用:

    import subprocess 
    import shlex 
     
    # ... your other imports and code... 
     
    _PROC_TIMEOUT_SEC = 5 
    SSH_COMMAND = ( 
                    f"ssh {SERVER_USER}@{SERVER} 'cd {SERVER_PERF_MEASUREMENT_SCRIPT_PATH}&&./" 
                    f"{SERVER_PERF_MEASUREMENT_SCRIPT} {MEASURE_TIME} {FILENAME}'" 
                  ) 
    AB_COMMAND = f"ab -c {CONCURRENCY} -t {TEST_TIME} -n {REQUESTS} {SERVER}{FILE_PATH}" 
    SCP_COMMAND = f"scp {SERVER_USER}@{SERVER}:{SERVER_PERF_MEASUREMENT_SCRIPT_PATH}/{LOG_FILE} ." 
     
    print("\nStarting the logging server side...\n") 
    portioned_ssh_command = shlex.split(SSH_COMMAND) 
    ssh_proc = subprocess.Popen(portioned_ssh_command) 
     
    # Still perform your testing as you had planned... 
    # Not sure if you need to sleep here, but I'll leave that up to you. 
    print(f"\nSleeping {SLEEP_TIME} seconds to visualize load gain...\n") 
    time.sleep(SLEEP_TIME) 
    print("\nStarting benchmark...\n") 
    os.system(AB_COMMAND) 
     
    # Wait for SSH process to finish before attempting to scp log file. 
    ssh_return_code = None 
    while(ssh_return_code is None): 
        try: 
            ssh_return_code = ssh_proc.wait(_PROC_TIMEOUT_SEC) 
        except subprocess.TimeoutExpired: 
            pass 
     
    print(f"Info: SSH completed with exitcode {ssh_proc.returncode}") 
     
    # Now that SSH  
    print("\nRecovering log file from server...\n") 
    os.system(SCP_COMMAND) 
     
    # ... The rest of your code... 
    

    备注 :我没有测试上面的代码,但我非常有信心它会做你想要完成的事情。


    评论关闭
    IT序号网

    微信公众号号:IT虾米 (左侧二维码扫一扫)欢迎添加!