AWS Redshift:严重:非引导用户超出了连接限制“ 500”

希望您一切都好。

我们经常达到此限制。我们知道在Redshift中没有办法增加并发用户连接的500个限制。我们还知道某些视图(pg_user_info)提供有关用户实际限制的信息。

我们正在寻找本论坛中找不到的一些答案,以及根据您的经验提供的任何指导。

问题:

  • 重新创建具有更大EC2实例的集群是否会产生更高的极限值?
  • 将新节点添加到现有群集中是否会产生更高的限值?
  • 从应用程序开发的角度来看:为了发现或预测将达到此限制的情况,您会建议采取哪些具体策略/操作?

    Txs-吉米

  • mfj111071 回答:AWS Redshift:严重:非引导用户超出了连接限制“ 500”

    正如您所说,这是Redshift中的硬性限制,没有办法提高它。 Redshift不是高并发/高连接数据库。

    我希望,如果您需要Redshift的大数据分析能力,则可以通过连接共享解决此问题。 Pgpool是为此的常用工具。

    ,

    好的。
    感谢所有回答。
    我在AWS上发布了支持票,这是建议,将所有内容粘贴在这里,虽然很长,但我希望它对遇到此问题的许多人都有用。这个想法是要在情况发生之前及时发现:

    To monitor the number of connections made to the database,you can create a cloudwatch alarm based on the Database connections metrics that will trigger a lambda function when a certain threshold is reached. This lambda function can then terminate idle connections by calling a procedure that terminates idle connections.
    
    Please find the query that creates a procedure to log and terminate long running inactive sessions
    :
    
    1. Add view to get all current inactive sessions in the cluster
    
    CREATE OR REPLACE VIEW inactive_sessions as (
        select a.process,trim(a.user_name) as user_name,trim(c.remotehost) as remotehost,a.usesysid,a.starttime,datediff(s,sysdate) as session_dur,b.last_end,case when b.last_end is not null then b.last_end else a.starttime end,sysdate) idle_dur
            FROM
            (
                select starttime,process,u.usesysid,user_name 
                from stv_sessions s,pg_user u 
                where 
                s.user_name = u.usename 
                and u.usesysid>1
                and process NOT IN (select pid from stv_inflight where userid>1 
                union select pid from stv_recents where status != 'Done' and userid>1)
            ) a 
            LEFT OUTER JOIN (
                select 
                userid,pid,max(endtime) as last_end from svl_statementtext 
                where userid>1 and sequence=0 group by 1,2) b ON a.usesysid = b.userid AND a.process = b.pid
    
            LEFT OUTER JOIN (
                select username,remotehost from stl_connection_log
                where event = 'initiating session' and username <> 'rsdb') c on a.user_name = c.username AND a.process = c.pid
            WHERE (b.last_end > a.starttime OR b.last_end is null)
            ORDER BY idle_dur
    );
    
    2. Add table for logging information about long running transactions that was terminated 
    
    CREATE TABLE IF NOT EXISTS terminated_inactive_sessions (
        process int,user_name varchar(50),remotehost varchar(50),starttime timestamp,session_dur int,idle_dur int,terminated_on timestamp DEFAULT GETDATE()   
    );
    
    3. Add procedure to log and terminate any inactive transactions running for longer than 'n' amount of seconds
    
    CREATE OR REPLACE PROCEDURE terminate_and_log_inactive_sessions (n INTEGER) 
    AS $$ 
    DECLARE
      expired RECORD ; 
    BEGIN
    FOR expired IN SELECT process,user_name,remotehost,starttime,session_dur,idle_dur FROM inactive_sessions where idle_dur >= n
    LOOP
    EXECUTE 'INSERT INTO terminated_inactive_sessions (process,idle_dur) values (' || expired.process || ',''' || expired.user_name || ''',''' || expired.remotehost || ''',''' || expired.starttime || ''',' || expired.session_dur || ',' || expired.idle_dur || ');';
    EXECUTE 'SELECT PG_TERMINATE_BACKEND(' || expired.process || ')';
    END LOOP ; 
    
    END ; 
    $$ LANGUAGE plpgsql;
    
    4. Execute the procedure by running the following command:
    
      call terminate_and_log_inactive_sessions(100);
    
    
    Here is a sample lambda function that attempts to close idle connections by querying the view 'inactive_sessions' created above,which you can use as a reference. 
    
    #Current time
    now = datetime.datetime.now()
    
    query = "SELECT process,idle_dur FROM inactive_sessions where idle_dur >= %d"
    
    logger = logging.getLogger()
    logger.setLevel(logging.INFO)
    
    def lambda_handler(event,context):
    
       try:
           conn = psycopg2.connect("dbname=" + db_database + " user=" + db_user + " password=" + db_password + " port=" + db_port + " host=" + db_host)
           conn.autocommit = True
       except:
           logger.error("ERROR: Unexpected error: Could not connect to Redshift cluster.")   
           sys.exit()
    
       logger.info("SUCCESS: Connection to RDS Redshift cluster succeeded")
    
       with conn.cursor() as cur:
           cur.execute(query % (session_idle_limit))
           row_count = cur.rowcount
           if row_count >=1:
               result = cur.fetchall()
               for row in result:
                   print("terminating session with pid %s that has been idle for %d seconds at %s" % (row[0],row[3],now))
                  cur.execute("SELECT PG_TERMINATE_BACKEND(%s);" % (row[0]))
               conn.close()
           else:
               conn.close()
    
    本文链接:https://www.f2er.com/2417850.html

    大家都在问