osmoappdesc.py, tests: switch to python 3

Make build and external tests work with python3, so we can drop
the python2 dependency.

This should be merged shortly after osmo-python-tests was migrated to
python3, and the jenkins build slaves were (automatically) updated to
have the new osmo-python-tests installed.

Related: OS#2819
Depends: osmo-python-tests I3ffc3519bf6c22536a49dad7a966188ddad351a7
Change-Id: I8c07d99c1bc9f0383e4bce17544e0998998cc54d

.gitignore

@@ -39,9 +39,9 @@ ltmain.sh
 
 
 # apps and app data
-src/gprs/osmo-sgsn
-src/gprs/osmo-gbproxy
-src/gprs/osmo-gtphub
+src/sgsn/osmo-sgsn
+src/gbproxy/osmo-gbproxy
+src/gtphub/osmo-gtphub
 src/libcommon/gsup_test_client
 
 #tests

doc/manuals/Makefile.am

@@ -1,16 +1,21 @@
 EXTRA_DIST = osmosgsn-usermanual.adoc \
     osmosgsn-usermanual-docinfo.xml \
     osmosgsn-vty-reference.xml \
+    osmogbproxy-usermanual.adoc \
+    osmogbproxy-usermanual-docinfo.xml \
     regen_doc.sh \
     chapters \
-    vty
+    vty \
+    osmogbproxy-vty-reference.xml \
+    vty-osmogbproxy \
+    $(NULL)
 
 if BUILD_MANUALS
-  ASCIIDOC = osmosgsn-usermanual.adoc
+  ASCIIDOC = osmosgsn-usermanual.adoc osmogbproxy-usermanual.adoc
   ASCIIDOC_DEPS = $(srcdir)/chapters/*.adoc
   include $(OSMO_GSM_MANUALS_DIR)/build/Makefile.asciidoc.inc
 
-  VTY_REFERENCE = osmosgsn-vty-reference.xml
+  VTY_REFERENCE = osmosgsn-vty-reference.xml osmogbproxy-vty-reference.xml
   include $(OSMO_GSM_MANUALS_DIR)/build/Makefile.vty-reference.inc
 
   OSMO_REPOSITORY = osmo-sgsn

doc/manuals/chapters/gbproxy-configuration.adoc

@@ -0,0 +1,6 @@
+== Configuring OsmoGbPROXY
+
+TBD.  Unfortunately this chapter of the manual still needs to be written.
+Osmocom has very limited funding and support resources; Feel free to help
+us completing this documentation by contributing with code, documentation
+or by supporting the developers financially.

doc/manuals/chapters/gbproxy-control.adoc

@@ -0,0 +1,29 @@
+[[control]]
+== Control interface
+
+The actual protocol is described in <<common-control-if>>, the variables
+common to all programs using it are described in <<ctrl_common_vars>>. Here we
+describe variables specific to OsmoGbPROXY.
+
+.Variables available over control interface
+[options="header",width="100%",cols="20%,5%,5%,50%,20%"]
+|===
+|Name|Access|Trap|Value|Comment
+|nsvc-state|RO|No|"<nsei>,<nsvci>,<local-alive>,<local-blocked>,<remote-role>,<remote-alive>,<remote-blocked>"|See <<nsvc_state>> for details.
+|gbproxy-state|RO|No|"<nsei>,<bvci>,<mcc>,<mnc>,<lac>,<rac>,<blocked>"|See <<gbproxy_state>> for details.
+|number-of-peers|RO|No|"<num-of-bss>"|Count of concurrent BSS(BTS) peers.
+|===
+
+[[nsvc_state]]
+=== nsvc-state
+
+Return the list of active NS-VCs (NS Virtual Circuits), including information
+on the key parameters, such as NSEI, NSVCI and the local + remote ALIVE
+and BLOCKED state.
+
+[[gbproxy_state]]
+=== gbproxy-state
+
+Return the list of active Peers, including information on the key
+parameters, such as NSEI, BVCI, and the MCC-MNC-LAC-RAC of the attached
+BSS, as well as the overall state (BLOCKED or UNBLOCKED).

doc/manuals/chapters/gbproxy-overview.adoc

@@ -0,0 +1,127 @@
+[[chapter_overview]]
+== Overview
+
+=== About OsmoGbPROXY
+
+OsmoGbPROXY is the Osmocom proxy for the 3GPP Gb interface.  The Gb
+interface is defined by 3GPP as the protocol between the BSS and the
+SGSN inside the 2G/2.5G/2.75G packet switched network domain.
+
+As Osmocom implements a BTS-colocated PCU, there are potentially many
+Gb interface connections between all those many PCUs in the network
+and the SGSN.  This can be cumbersome to configure/maintain at the
+SGSN sine.
+
+OsmoGbPROXY aggregates many PCU-facing Gb connections into one Gb
+connection to the SGSN.  This is achieved by
+
+* maintaining sepaate NS-VCs on the PCU side and on the SGSN side
+* more or less transparently routing BSSGP peer-to-peer Virtual Circuits
+  (BVCs) through the proxy
+* having some special handling for the signaling BVC (BVCI=0) which is
+  shared among all the PCUs connected to the proxy
+
+=== Data Model
+
+==== gbproxy_config
+
+This contains the parsed configuration of the OsmoGbPROXY.
+
+==== gproxy_peer
+
+A "peer" is any remote NS-entity that the proxy interacts with.  A peer
+includes information about:
+
+* the [unique] NSEI of the peer
+* the [unique] BVCI of the peer
+* the Routeing Area (RA) of the peer
+
+==== gbproxy_tlli_state
+
+One of the (unique) TLLI of any of the subscribers/UEs attached to any of
+the BTSs/PCUs served by the proxy.
+
+==== gbproxy_link_info
+
+One of the [unique] subscribers/connections that are served through this
+proxy.  The information includes
+
+* the TLLI on BSS side
+* the TLLI on SGSN side (may be different due to P-TMSI rewriting)
+* the NSEI of the SGSN for this link
+* a timestamp when we last conversed with this subscriber
+* state related to IMSI acquisition
+** a temporary queue of stored messages (until IMSI acquisition succeeds)
+** N(U) rewriting state (inserting IDENTTIY REQ changes LLC sequence numbers)
+
+==== gbproxy_match
+
+A single matching rule against which IMSIs are matched.  The matching rule
+is expressed as regular expression.  There can be one such matching rule for
+each
+
+* routing between two different SGSNs, see below
+* patching of messages (e.g. APN, PLMN)
+
+
+=== Advanced Features
+
+==== PLMN patching
+
+This feature permits to modify the PLMN inside any BSSGP messages
+containing the Routing Area ID (RAID).
+
+The configured core-mcc and core-mnc will be used towards the SGSN,
+irrespective of which MCC/MNC the PCU is using/reporting on Gb.
+
+==== APN patching
+
+This will transparently re-write the APN name inside SM ACTIVATE PDP
+REQUEST messages on the way from the MS to the SGSN.  The patching is
+performed based on matching on the IMSI of the subscriber.
+
+The configured core-apn will be used towards the SGSN, irrespective
+of which APN the MS is requesting in its Layer3 signaling.
+
+APN patching can only be performed if no GPRS encryption is enabled in
+the network!
+
+APN patching is useful in case a valid APN cannot reliably be
+provisioned via other means, such as via the SIM Card, OTA-DM or via
+CAMEL rewriting in the SGSN.
+
+==== P-TMSI patching
+
+This feature transparently rewrite the P-TMSI between MS and SGSN.  This
+is required when using the Secondary SGSN support, as both SGSNs could
+allocate overlapping TMSIs and we must make sure they're unique across
+both SGSNs.
+
+P-TMSI patching is required by (and hence automatically enablede if
+secondary SGSN support is enabled.
+
+P-TMSI patching can only be performed if no GPRS encryption is enabled in
+the network!
+
+==== IMSI Acquisition
+
+This is a special feature where the proxy will by itself inject GMM IDENTITY
+REQUEST messages for the IMSI into the downlink BSSGP traffic in order
+to establish the IMSI of subscribers for which it is not otherwise known
+
+IMSI acquisition is automatically enabled if secondary SGSN support is
+enabled.
+
+==== Secondary SGSN Support
+
+This allows the proxy to connect not only to one SGSN, but to two
+different SGSNs.  IMSI matching rules are applied to determine which of
+the SGSNs is to be used for traffic of this subscriber.
+
+One possible use case of this feature is to have a "local break-out" for
+subscribers who are native to this network (and hence save
+latencies/overhead of back-hauling all related traffic via the
+SGSN+GGSN) while at the same time maintaining the classic behavior for
+inbound roaming subscribers, where the roaming agreements mandate that
+data traffic is brought back to the GGSN in the HPLMN via the SGSN of
+the VPLMN.

doc/manuals/chapters/gbproxy-running.adoc

@@ -0,0 +1,39 @@
+== Running OsmoGbPROXY
+
+The OsmoGbPROXY executable (`osmo-gbproxy`) offers the following command-line
+options:
+
+
+=== SYNOPSIS
+
+*osmo-gbproxy* [-h|-V] [-d 'DBGMASK'] [-D] [-c 'CONFIGFILE'] [-s] [-e 'LOGLEVEL'] [-T]
+
+
+=== OPTIONS
+
+*-h, --help*::
+	Print a short help message about the supported options
+*-V, --version*::
+	Print the compile-time version number of the program
+*-d, --debug 'DBGMASK','DBGLEVELS'*::
+	Set the log subsystems and levels for logging to stderr. This
+	has mostly been superseded by VTY-based logging configuration,
+	see <<logging>> for further information.
+*-D, --daemonize*::
+	Fork the process as a daemon into background.
+*-c, --config-file 'CONFIGFILE'*::
+	Specify the file and path name of the configuration file to be
+	used. If none is specified, use `osmo_sgsn.cfg` in the current
+	working directory.
+*-s, --disable-color*::
+	Disable colors for logging to stderr. This has mostly been
+	deprecated by VTY based logging configuration, see <<logging>>
+	for more information.
+*-e, --log-level 'LOGLEVEL'*::
+	Set the global log level for logging to stderr. This has mostly
+	been deprecated by VTY based logging configuration, see
+	<<logging>> for more information.
+*-T, --timestamp*::
+        Enable prefixing each log line on stderr with a timestamp.  This
+        has mostly been deprecated by VTY based logging configuration, see
+        <<logging>> for more information.

doc/manuals/osmogbproxy-usermanual-docinfo.xml

@@ -0,0 +1,46 @@
+<revhistory>
+  <revision>
+    <revnumber>1</revnumber>
+    <date>March 21, 2019</date>
+    <authorinitials>HW</authorinitials>
+    <revremark>
+      Initial version.
+    </revremark>
+  </revision>
+</revhistory>
+
+<authorgroup>
+  <author>
+    <firstname>Harald</firstname>
+    <surname>Welte</surname>
+    <email>hwelte@sysmocom.de</email>
+    <authorinitials>HW</authorinitials>
+    <affiliation>
+      <shortaffil>sysmocom</shortaffil>
+      <orgname>sysmocom - s.f.m.c. GmbH</orgname>
+      <jobtitle>Managing Director</jobtitle>
+    </affiliation>
+  </author>
+</authorgroup>
+
+<copyright>
+  <year>2013-2019</year>
+  <holder>sysmocom - s.f.m.c. GmbH</holder>
+</copyright>
+
+<legalnotice>
+  <para>
+	Permission is granted to copy, distribute and/or modify this
+	document under the terms of the GNU Free Documentation License,
+	Version 1.3 or any later version published by the Free Software
+	Foundation; with no Invariant Sections, no Front-Cover Texts,
+	and no Back-Cover Texts.  A copy of the license is included in
+	the section entitled "GNU Free Documentation License".
+  </para>
+  <para>
+	The Asciidoc source code of this manual can be found at
+	<ulink url="https://git.osmocom.org/osmo-sgsn/doc/">
+		https://git.osmocom.org/osmo-sgsn/doc/
+	</ulink>
+  </para>
+</legalnotice>

doc/manuals/osmogbproxy-usermanual.adoc

@@ -0,0 +1,34 @@
+:gfdl-enabled:
+
+OsmoGbPROXY User Manual
+=======================
+Harald Welte <hwelte@sysmocom.de>
+
+
+include::./common/chapters/preface.adoc[]
+
+include::{srcdir}/chapters/gbproxy-overview.adoc[]
+
+include::{srcdir}/chapters/gbproxy-running.adoc[]
+
+include::{srcdir}/chapters/gbproxy-control.adoc[]
+
+include::./common/chapters/vty.adoc[]
+
+include::./common/chapters/logging.adoc[]
+
+include::{srcdir}/chapters/gbproxy-configuration.adoc[]
+
+include::./common/chapters/gb.adoc[]
+
+include::./common/chapters/control_if.adoc[]
+
+//include::{srcdir}/chapters/counters.adoc[]
+
+include::./common/chapters/port_numbers.adoc[]
+
+include::./common/chapters/bibliography.adoc[]
+
+include::./common/chapters/glossary.adoc[]
+
+include::./common/chapters/gfdl.adoc[]

doc/manuals/osmogbproxy-vty-reference.xml

@@ -0,0 +1,38 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+  ex:ts=2:sw=42sts=2:et
+  -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*-
+-->
+<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML 5.0//EN"
+"http://docbook.org/xml/5.0/dtd/docbook.dtd" [
+<!ENTITY chapter-vty      SYSTEM      "./common/chapters/vty.xml" >
+<!ENTITY sections-vty     SYSTEM      "generated/docbook_osmogbproxy-vty-reference.xml"  >
+]>
+
+<book>
+  <info>
+    <revhistory>
+        <revision>
+            <revnumber>v1</revnumber>
+            <date>2nd December 2019</date>
+            <authorinitials>hw</authorinitials>
+            <revremark>Initial</revremark>
+        </revision>
+    </revhistory>
+
+    <title>OsmoGbProxy VTY Reference</title>
+
+    <copyright>
+      <year>2019</year>
+    </copyright>
+
+    <legalnotice>
+      <para>This work is copyright by <orgname>sysmocom - s.f.m.c. GmbH</orgname>. All rights reserved.
+      </para>
+    </legalnotice>
+  </info>
+
+  <!-- Main chapters-->
+  &chapter-vty;
+</book>
+

doc/manuals/regen_doc.sh

@@ -1,17 +1,77 @@
-#!/bin/sh -x
+#!/bin/sh -e
 
-if [ -z "$DOCKER_PLAYGROUND" ]; then
-	echo "You need to set DOCKER_PLAYGROUND"
+require_osmo_interact_vty() {
+	if command -v osmo_interact_vty.py >/dev/null 2>&1; then
+		return
+	fi
+	echo "ERROR: osmo_interact_vty.py not found. Are osmo-python-tests in PATH?"
 	exit 1
-fi
+}
 
-SCRIPT=$(realpath "$0")
-MANUAL_DIR=$(dirname "$SCRIPT")
+# $1: "update_vty_reference" or "update_counters"
+# $2: output file
+# $3: port
+# $4-$n: command
+interact_vty() {
+	action="$1"
+	output="$2"
+	port="$3"
+	log="/tmp/$4.log"
+	shift 3
 
-COMMIT=${COMMIT:-$(git log -1 --format=format:%H)}
+	echo "Starting in background: $@"
+	"$@" > "$log" 2>&1 &
+	pid="$!"
 
-cd "$DOCKER_PLAYGROUND/scripts" || exit 1
+	sleep 0.5
+	if ! kill -0 "$pid" 2>/dev/null; then
+		echo "ERROR: start failed!"
+		cat "$log"
+		exit 1
+	fi
 
-OSMO_SGSN_BRANCH=$COMMIT ./regen_doc.sh osmo-sgsn 4245 \
-	"$MANUAL_DIR/chapters/counters_generated.adoc" \
-	"$MANUAL_DIR/vty/sgsn_vty_reference.xml"
+	case "$action" in
+		"update_vty_reference")
+			echo "Updating VTY reference: $output"
+			osmo_interact_vty.py -X -p "$port" -H 127.0.0.1 -O "$output"
+			;;
+		"update_counters")
+			echo "Updating asciidoc counters: $output"
+			osmo_interact_vty.py -c "enable;show asciidoc counters" -p "$port" -H 127.0.0.1 -O "$output"
+			;;
+		*)
+			echo "ERROR: invalid argument: $action"
+			exit 1
+			;;
+	esac
+
+	kill "$pid"
+	echo "Done (killed $1)"
+	echo
+}
+
+DIR="$(cd "$(dirname "$0")"; pwd)"
+cd "$DIR"
+
+require_osmo_interact_vty
+
+interact_vty \
+	"update_vty_reference" \
+	"vty/sgsn_vty_reference.xml" \
+	4245 \
+	osmo-sgsn -c "../examples/osmo-sgsn/osmo-sgsn.cfg"
+
+interact_vty \
+	"update_vty_reference" \
+	"vty-osmogbproxy/gbproxy_vty_reference.xml" \
+	4246 \
+	osmo-gbproxy -c "../examples/osmo-gbproxy/osmo-gbproxy.cfg"
+
+interact_vty \
+	"update_counters" \
+	"chapters/counters_generated.adoc" \
+	4245 \
+	osmo-sgsn -c "../examples/osmo-sgsn/osmo-sgsn.cfg"
+
+
+echo "Done with all"

doc/manuals/vty-osmogbproxy/gbproxy_vty_additions.xml

@@ -0,0 +1,5 @@
+<vtydoc xmlns='urn:osmocom:xml:libosmocore:vty:doc:1.0'>
+    <node id='config-gbproxy'>
+	    <description>Configure the Gb proxy</description>
+    </node>
+</vtydoc>

include/osmocom/sgsn/gprs_sgsn.h

@@ -428,7 +428,7 @@ int sgsn_ctrl_cmds_install(void);
  */
 struct imsi_acl_entry {
 	struct llist_head list;
-	char imsi[16+1];
+	char imsi[OSMO_IMSI_BUF_SIZE];
 };
 
 /* see GSM 09.02, 17.7.1, PDP-Context and GPRSSubscriptionData */

osmoappdesc.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 
 # (C) 2013 by Katerina Barone-Adesi <kat.obsc@gmail.com>
 # This program is free software: you can redistribute it and/or modify

src/gbproxy/gb_proxy_main.c

@@ -196,6 +196,11 @@ static void handle_options(int argc, char **argv)
 			break;
 		}
 	}
+
+	if (argc > optind) {
+		fprintf(stderr, "Unsupported positional arguments on command line\n");
+		exit(2);
+	}
 }
 
 int gbproxy_vty_is_config_node(struct vty *vty, int node)

src/gtphub/gtphub_main.c

@@ -322,6 +322,11 @@ static void handle_options(struct cmdline_cfg *ccfg, int argc, char **argv)
 			break;
 		}
 	}
+
+	if (argc > optind) {
+		fprintf(stderr, "Unsupported positional arguments on command line\n");
+		exit(2);
+	}
 }
 
 int main(int argc, char **argv)

src/sgsn/gprs_gmm.c

@@ -1626,7 +1626,7 @@ static int gsm48_rx_gmm_ra_upd_req(struct sgsn_mm_ctx *mmctx, struct msgb *msg,
 			/* send a XID reset to re-set all LLC sequence numbers
 			 * in the MS */
 			LOGGBP(llme, DMM, LOGL_NOTICE, "LLC XID RESET\n");
-			gprs_llgmm_reset(llme);
+			gprs_llgmm_reset_oldmsg(msg, GPRS_SAPI_GMM, llme);
 		}
 		/* The MS has to perform GPRS attach */
 		/* Device is still IMSI attached for CS but initiate GPRS ATTACH,

src/sgsn/gprs_gmm_attach.c

@@ -449,7 +449,7 @@ struct osmo_fsm gmm_attach_req_fsm = {
 
 static __attribute__((constructor)) void gprs_gmm_fsm_init(void)
 {
-	osmo_fsm_register(&gmm_attach_req_fsm);
+	OSMO_ASSERT(osmo_fsm_register(&gmm_attach_req_fsm) == 0);
 }
 
 void gmm_att_req_free(struct sgsn_mm_ctx *mm) {

src/sgsn/gprs_gmm_fsm.c

@@ -183,5 +183,5 @@ struct osmo_fsm gmm_fsm = {
 
 static __attribute__((constructor)) void gmm_fsm_init(void)
 {
-	osmo_fsm_register(&gmm_fsm);
+	OSMO_ASSERT(osmo_fsm_register(&gmm_fsm) == 0);
 }

src/sgsn/gprs_llc.c

@@ -69,17 +69,20 @@ static int gprs_llc_generate_xid(uint8_t *bytes, int bytes_len,
 	struct gprs_llc_xid_field xid_version;
 	struct gprs_llc_xid_field xid_n201u;
 	struct gprs_llc_xid_field xid_n201i;
+	uint16_t n201_u, n201_i;
 
 	xid_version.type = GPRS_LLC_XID_T_VERSION;
 	xid_version.data = (uint8_t *) "\x00";
 	xid_version.data_len = 1;
 
+	n201_u = htons(lle->params.n201_u);
 	xid_n201u.type = GPRS_LLC_XID_T_N201_U;
-	xid_n201u.data = (uint8_t *) "\x05\xf0";
+	xid_n201u.data = (uint8_t *) &n201_u;
 	xid_n201u.data_len = 2;
 
+	n201_i = htons(lle->params.n201_i);
 	xid_n201i.type = GPRS_LLC_XID_T_N201_I;
-	xid_n201i.data = (uint8_t *) "\x05\xf0";
+	xid_n201i.data = (uint8_t *) &n201_i;
 	xid_n201i.data_len = 2;
 
 	/* Add locally managed XID Fields */

src/sgsn/gprs_mm_state_gb_fsm.c

@@ -125,5 +125,5 @@ struct osmo_fsm mm_state_gb_fsm = {
 
 static __attribute__((constructor)) void mm_state_gb_fsm_init(void)
 {
-	osmo_fsm_register(&mm_state_gb_fsm);
+	OSMO_ASSERT(osmo_fsm_register(&mm_state_gb_fsm) == 0);
 }

src/sgsn/gprs_mm_state_iu_fsm.c

@@ -146,5 +146,5 @@ struct osmo_fsm mm_state_iu_fsm = {
 
 static __attribute__((constructor)) void mm_state_iu_fsm_init(void)
 {
-	osmo_fsm_register(&mm_state_iu_fsm);
+	OSMO_ASSERT(osmo_fsm_register(&mm_state_iu_fsm) == 0);
 }

src/sgsn/sgsn_main.c

@@ -271,6 +271,11 @@ static void handle_options(int argc, char **argv)
 			break;
 		}
 	}
+
+	if (argc > optind) {
+		fprintf(stderr, "Unsupported positional arguments on command line\n");
+		exit(2);
+	}
 }
 
 /* default categories */
@@ -393,7 +398,7 @@ int main(int argc, char **argv)
 	ctrl_vty_init(tall_sgsn_ctx);
 
 #if BUILD_IU
-	osmo_ss7_init();
+	OSMO_ASSERT(osmo_ss7_init() == 0);
 	osmo_ss7_vty_init_asp(tall_sgsn_ctx);
 	osmo_sccp_vty_init();
 #endif

tests/ctrl_test_runner.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python3
 
 # (C) 2013 by Jacob Erlbeck <jerlbeck@sysmocom.de>
 # (C) 2014 by Holger Hans Peter Freyther
@@ -53,8 +53,8 @@ class TestCtrlBase(unittest.TestCase):
         try:
             self.proc = osmoutil.popen_devnull(osmo_ctrl_cmd)
         except OSError:
-            print >> sys.stderr, "Current directory: %s" % os.getcwd()
-            print >> sys.stderr, "Consider setting -b"
+            print("Current directory: %s" % os.getcwd(), file=sys.stderr)
+            print("Consider setting -b", file=sys.stderr)
         time.sleep(2)
 
         appstring = self.ctrl_app()[2]
@@ -72,7 +72,7 @@ class TestCtrlBase(unittest.TestCase):
 
     def connect(self, host, port):
         if verbose:
-            print "Connecting to host %s:%i" % (host, port)
+            print("Connecting to host %s:%i" % (host, port))
 
         retries = 30
         while True:
@@ -92,7 +92,7 @@ class TestCtrlBase(unittest.TestCase):
 
     def send(self, data):
         if verbose:
-            print "Sending \"%s\"" %(data)
+            print("Sending \"%s\"" %(data))
         data = Ctrl().add_header(data)
         return self.sock.send(data) == len(data)
 
@@ -121,9 +121,9 @@ class TestCtrlBase(unittest.TestCase):
         data = self.sock.recv(4096)
         while (len(data)>0):
             (head, data) = IPA().split_combined(data)
-            answer = Ctrl().rem_header(head)
+            answer = Ctrl().rem_header(head).decode()
             if verbose:
-                print "Got message:", answer
+                print("Got message:", answer)
             (mtype, id, msg) = answer.split(None, 2)
             id = int(id)
             rsp = {'mtype': mtype, 'id': id}
@@ -139,7 +139,7 @@ class TestCtrlBase(unittest.TestCase):
             responses[id] = rsp
 
         if verbose:
-            print "Decoded replies: ", responses
+            print("Decoded replies: ", responses)
 
         return responses
 
@@ -154,9 +154,9 @@ class TestCtrlSGSN(TestCtrlBase):
     def testListSubscribers(self):
         # TODO. Add command to mark a subscriber as active
         r = self.do_get('subscriber-list-active-v1')
-        self.assertEquals(r['mtype'], 'GET_REPLY')
-        self.assertEquals(r['var'], 'subscriber-list-active-v1')
-        self.assertEquals(r['value'], None)
+        self.assertEqual(r['mtype'], 'GET_REPLY')
+        self.assertEqual(r['var'], 'subscriber-list-active-v1')
+        self.assertEqual(r['value'], None)
 
 def add_sgsn_test(suite, workdir):
     if not os.path.isfile(os.path.join(workdir, "src/sgsn/osmo-sgsn")):
@@ -191,9 +191,9 @@ if __name__ == '__main__':
     if args.p:
         confpath = args.p
 
-    print "confpath %s, workdir %s" % (confpath, workdir)
+    print("confpath %s, workdir %s" % (confpath, workdir))
     os.chdir(workdir)
-    print "Running tests for specific control commands"
+    print("Running tests for specific control commands")
     suite = unittest.TestSuite()
     add_sgsn_test(suite, workdir)
     res = unittest.TextTestRunner(verbosity=verbose_level).run(suite)

tests/vty_test_runner.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python3
 
 # (C) 2013 by Katerina Barone-Adesi <kat.obsc@gmail.com>
 # (C) 2013 by Holger Hans Peter Freyther
@@ -33,9 +33,9 @@ class TestVTYBase(unittest.TestCase):
     def checkForEndAndExit(self):
         res = self.vty.command("list")
         #print ('looking for "exit"\n')
-        self.assert_(res.find('  exit\r') > 0)
+        self.assertTrue(res.find('  exit\r') > 0)
         #print 'found "exit"\nlooking for "end"\n'
-        self.assert_(res.find('  end\r') > 0)
+        self.assertTrue(res.find('  end\r') > 0)
         #print 'found "end"\n'
 
     def vty_command(self):
@@ -54,8 +54,8 @@ class TestVTYBase(unittest.TestCase):
         try:
             self.proc = osmoutil.popen_devnull(osmo_vty_cmd)
         except OSError:
-            print >> sys.stderr, "Current directory: %s" % os.getcwd()
-            print >> sys.stderr, "Consider setting -b"
+            print("Current directory: %s" % os.getcwd(), file=sys.stderr)
+            print("Consider setting -b", file=sys.stderr)
 
         appstring = self.vty_app()[2]
         appport = self.vty_app()[0]
@@ -80,41 +80,41 @@ class TestVTYGbproxy(TestVTYBase):
     def testVtyTree(self):
         self.vty.enable()
         self.assertTrue(self.vty.verify('configure terminal', ['']))
-        self.assertEquals(self.vty.node(), 'config')
+        self.assertEqual(self.vty.node(), 'config')
         self.checkForEndAndExit()
         self.assertTrue(self.vty.verify('ns', ['']))
-        self.assertEquals(self.vty.node(), 'config-ns')
+        self.assertEqual(self.vty.node(), 'config-ns')
         self.checkForEndAndExit()
         self.assertTrue(self.vty.verify('exit', ['']))
-        self.assertEquals(self.vty.node(), 'config')
+        self.assertEqual(self.vty.node(), 'config')
         self.assertTrue(self.vty.verify('gbproxy', ['']))
-        self.assertEquals(self.vty.node(), 'config-gbproxy')
+        self.assertEqual(self.vty.node(), 'config-gbproxy')
         self.checkForEndAndExit()
         self.assertTrue(self.vty.verify('exit', ['']))
-        self.assertEquals(self.vty.node(), 'config')
+        self.assertEqual(self.vty.node(), 'config')
 
     def testVtyShow(self):
         res = self.vty.command("show ns")
-        self.assert_(res.find('Encapsulation NS-UDP-IP') >= 0)
+        self.assertTrue(res.find('Encapsulation NS-UDP-IP') >= 0)
 
         res = self.vty.command("show gbproxy stats")
-        self.assert_(res.find('GBProxy Global Statistics') >= 0)
+        self.assertTrue(res.find('GBProxy Global Statistics') >= 0)
 
     def testVtyDeletePeer(self):
         self.vty.enable()
         self.assertTrue(self.vty.verify('delete-gbproxy-peer 9999 bvci 7777', ['BVC not found']))
         res = self.vty.command("delete-gbproxy-peer 9999 all dry-run")
-        self.assert_(res.find('Not Deleted 0 BVC') >= 0)
-        self.assert_(res.find('Not Deleted 0 NS-VC') >= 0)
+        self.assertTrue(res.find('Not Deleted 0 BVC') >= 0)
+        self.assertTrue(res.find('Not Deleted 0 NS-VC') >= 0)
         res = self.vty.command("delete-gbproxy-peer 9999 only-bvc dry-run")
-        self.assert_(res.find('Not Deleted 0 BVC') >= 0)
-        self.assert_(res.find('Not Deleted 0 NS-VC') < 0)
+        self.assertTrue(res.find('Not Deleted 0 BVC') >= 0)
+        self.assertTrue(res.find('Not Deleted 0 NS-VC') < 0)
         res = self.vty.command("delete-gbproxy-peer 9999 only-nsvc dry-run")
-        self.assert_(res.find('Not Deleted 0 BVC') < 0)
-        self.assert_(res.find('Not Deleted 0 NS-VC') >= 0)
+        self.assertTrue(res.find('Not Deleted 0 BVC') < 0)
+        self.assertTrue(res.find('Not Deleted 0 NS-VC') >= 0)
         res = self.vty.command("delete-gbproxy-peer 9999 all")
-        self.assert_(res.find('Deleted 0 BVC') >= 0)
-        self.assert_(res.find('Deleted 0 NS-VC') >= 0)
+        self.assertTrue(res.find('Deleted 0 BVC') >= 0)
+        self.assertTrue(res.find('Deleted 0 NS-VC') >= 0)
 
 class TestVTYSGSN(TestVTYBase):
 
@@ -128,22 +128,22 @@ class TestVTYSGSN(TestVTYBase):
     def testVtyTree(self):
         self.vty.enable()
         self.assertTrue(self.vty.verify('configure terminal', ['']))
-        self.assertEquals(self.vty.node(), 'config')
+        self.assertEqual(self.vty.node(), 'config')
         self.checkForEndAndExit()
         self.assertTrue(self.vty.verify('ns', ['']))
-        self.assertEquals(self.vty.node(), 'config-ns')
+        self.assertEqual(self.vty.node(), 'config-ns')
         self.checkForEndAndExit()
         self.assertTrue(self.vty.verify('exit', ['']))
-        self.assertEquals(self.vty.node(), 'config')
+        self.assertEqual(self.vty.node(), 'config')
         self.assertTrue(self.vty.verify('sgsn', ['']))
-        self.assertEquals(self.vty.node(), 'config-sgsn')
+        self.assertEqual(self.vty.node(), 'config-sgsn')
         self.checkForEndAndExit()
         self.assertTrue(self.vty.verify('exit', ['']))
-        self.assertEquals(self.vty.node(), 'config')
+        self.assertEqual(self.vty.node(), 'config')
 
     def testVtyShow(self):
         res = self.vty.command("show ns")
-        self.assert_(res.find('Encapsulation NS-UDP-IP') >= 0)
+        self.assertTrue(res.find('Encapsulation NS-UDP-IP') >= 0)
         self.assertTrue(self.vty.verify('show bssgp', ['']))
         self.assertTrue(self.vty.verify('show bssgp stats', ['']))
         self.assertTrue(self.vty.verify('show bssgp nsei 123', ['']))
@@ -154,57 +154,57 @@ class TestVTYSGSN(TestVTYBase):
         self.assertTrue(self.vty.verify('show pdp-context all', ['']))
 
         res = self.vty.command("show sndcp")
-        self.assert_(res.find('State of SNDCP Entities') >= 0)
+        self.assertTrue(res.find('State of SNDCP Entities') >= 0)
 
         res = self.vty.command("show llc")
-        self.assert_(res.find('State of LLC Entities') >= 0)
+        self.assertTrue(res.find('State of LLC Entities') >= 0)
 
     def testVtyAuth(self):
         self.vty.enable()
         self.assertTrue(self.vty.verify('configure terminal', ['']))
-        self.assertEquals(self.vty.node(), 'config')
+        self.assertEqual(self.vty.node(), 'config')
         self.assertTrue(self.vty.verify('sgsn', ['']))
-        self.assertEquals(self.vty.node(), 'config-sgsn')
+        self.assertEqual(self.vty.node(), 'config-sgsn')
         self.assertTrue(self.vty.verify('auth-policy accept-all', ['']))
         res = self.vty.command("show running-config")
-        self.assert_(res.find('auth-policy accept-all') > 0)
+        self.assertTrue(res.find('auth-policy accept-all') > 0)
         self.assertTrue(self.vty.verify('auth-policy acl-only', ['']))
         res = self.vty.command("show running-config")
-        self.assert_(res.find('auth-policy acl-only') > 0)
+        self.assertTrue(res.find('auth-policy acl-only') > 0)
         self.assertTrue(self.vty.verify('auth-policy closed', ['']))
         res = self.vty.command("show running-config")
-        self.assert_(res.find('auth-policy closed') > 0)
+        self.assertTrue(res.find('auth-policy closed') > 0)
         self.assertTrue(self.vty.verify('gsup remote-ip 127.0.0.4', ['']))
         self.assertTrue(self.vty.verify('gsup remote-port 2222', ['']))
         self.assertTrue(self.vty.verify('auth-policy remote', ['']))
         res = self.vty.command("show running-config")
-        self.assert_(res.find('auth-policy remote') > 0)
+        self.assertTrue(res.find('auth-policy remote') > 0)
 
     def testVtySubscriber(self):
         self.vty.enable()
         res = self.vty.command('show subscriber cache')
-        self.assert_(res.find('1234567890') < 0)
+        self.assertTrue(res.find('1234567890') < 0)
         self.assertTrue(self.vty.verify('update-subscriber imsi 1234567890 create', ['']))
         res = self.vty.command('show subscriber cache')
-        self.assert_(res.find('1234567890') >= 0)
-        self.assert_(res.find('Authorized: 0') >= 0)
+        self.assertTrue(res.find('1234567890') >= 0)
+        self.assertTrue(res.find('Authorized: 0') >= 0)
         self.assertTrue(self.vty.verify('update-subscriber imsi 1234567890 update-location-result ok', ['']))
         res = self.vty.command('show subscriber cache')
-        self.assert_(res.find('1234567890') >= 0)
-        self.assert_(res.find('Authorized: 1') >= 0)
+        self.assertTrue(res.find('1234567890') >= 0)
+        self.assertTrue(res.find('Authorized: 1') >= 0)
         self.assertTrue(self.vty.verify('update-subscriber imsi 1234567890 cancel update-procedure', ['']))
         res = self.vty.command('show subscriber cache')
-        self.assert_(res.find('1234567890') >= 0)
+        self.assertTrue(res.find('1234567890') >= 0)
         self.assertTrue(self.vty.verify('update-subscriber imsi 1234567890 destroy', ['']))
         res = self.vty.command('show subscriber cache')
-        self.assert_(res.find('1234567890') < 0)
+        self.assertTrue(res.find('1234567890') < 0)
 
     def testVtyGgsn(self):
         self.vty.enable()
         self.assertTrue(self.vty.verify('configure terminal', ['']))
-        self.assertEquals(self.vty.node(), 'config')
+        self.assertEqual(self.vty.node(), 'config')
         self.assertTrue(self.vty.verify('sgsn', ['']))
-        self.assertEquals(self.vty.node(), 'config-sgsn')
+        self.assertEqual(self.vty.node(), 'config-sgsn')
         self.assertTrue(self.vty.verify('ggsn 0 remote-ip 127.99.99.99', ['']))
         self.assertTrue(self.vty.verify('ggsn 0 gtp-version 1', ['']))
         self.assertTrue(self.vty.verify('apn * ggsn 0', ['']))
@@ -213,65 +213,65 @@ class TestVTYSGSN(TestVTYBase):
         self.assertTrue(self.vty.verify('apn apn1.test imsi-prefix 123456 ggsn 0', ['']))
         self.assertTrue(self.vty.verify('apn apn2.test imsi-prefix 123456 ggsn 0', ['']))
         res = self.vty.command("show running-config")
-        self.assert_(res.find('ggsn 0 remote-ip 127.99.99.99') >= 0)
-        self.assert_(res.find('ggsn 0 gtp-version 1') >= 0)
-        self.assert_(res.find('apn * ggsn 0') >= 0)
-        self.assert_(res.find('apn apn1.test ggsn 0') >= 0)
-        self.assert_(res.find('apn apn1.test imsi-prefix 123456 ggsn 0') >= 0)
-        self.assert_(res.find('apn apn2.test imsi-prefix 123456 ggsn 0') >= 0)
+        self.assertTrue(res.find('ggsn 0 remote-ip 127.99.99.99') >= 0)
+        self.assertTrue(res.find('ggsn 0 gtp-version 1') >= 0)
+        self.assertTrue(res.find('apn * ggsn 0') >= 0)
+        self.assertTrue(res.find('apn apn1.test ggsn 0') >= 0)
+        self.assertTrue(res.find('apn apn1.test imsi-prefix 123456 ggsn 0') >= 0)
+        self.assertTrue(res.find('apn apn2.test imsi-prefix 123456 ggsn 0') >= 0)
 
     def testVtyEasyAPN(self):
         self.vty.enable()
         self.assertTrue(self.vty.verify('configure terminal', ['']))
-        self.assertEquals(self.vty.node(), 'config')
+        self.assertEqual(self.vty.node(), 'config')
         self.assertTrue(self.vty.verify('sgsn', ['']))
-        self.assertEquals(self.vty.node(), 'config-sgsn')
+        self.assertEqual(self.vty.node(), 'config-sgsn')
 
         res = self.vty.command("show running-config")
-        self.assertEquals(res.find("apn internet"), -1)
+        self.assertEqual(res.find("apn internet"), -1)
 
         self.assertTrue(self.vty.verify("access-point-name internet.apn", ['']))
         res = self.vty.command("show running-config")
-        self.assert_(res.find("apn internet.apn ggsn 0") >= 0)
+        self.assertTrue(res.find("apn internet.apn ggsn 0") >= 0)
 
         self.assertTrue(self.vty.verify("no access-point-name internet.apn", ['']))
         res = self.vty.command("show running-config")
-        self.assertEquals(res.find("apn internet"), -1)
+        self.assertEqual(res.find("apn internet"), -1)
 
     def testVtyCDR(self):
         self.vty.enable()
         self.assertTrue(self.vty.verify('configure terminal', ['']))
-        self.assertEquals(self.vty.node(), 'config')
+        self.assertEqual(self.vty.node(), 'config')
         self.assertTrue(self.vty.verify('sgsn', ['']))
-        self.assertEquals(self.vty.node(), 'config-sgsn')
+        self.assertEqual(self.vty.node(), 'config-sgsn')
 
         res = self.vty.command("show running-config")
-        self.assert_(res.find("no cdr filename") > 0)
+        self.assertTrue(res.find("no cdr filename") > 0)
 
         self.vty.command("cdr filename bla.cdr")
         res = self.vty.command("show running-config")
-        self.assertEquals(res.find("no cdr filename"), -1)
-        self.assert_(res.find(" cdr filename bla.cdr") > 0)
+        self.assertEqual(res.find("no cdr filename"), -1)
+        self.assertTrue(res.find(" cdr filename bla.cdr") > 0)
 
         self.vty.command("no cdr filename")
         res = self.vty.command("show running-config")
-        self.assert_(res.find("no cdr filename") > 0)
-        self.assertEquals(res.find(" cdr filename bla.cdr"), -1)
+        self.assertTrue(res.find("no cdr filename") > 0)
+        self.assertEqual(res.find(" cdr filename bla.cdr"), -1)
 
         res = self.vty.command("show running-config")
-        self.assert_(res.find(" cdr interval 600") > 0)
+        self.assertTrue(res.find(" cdr interval 600") > 0)
 
         self.vty.command("cdr interval 900")
         res = self.vty.command("show running-config")
-        self.assert_(res.find(" cdr interval 900") > 0)
-        self.assertEquals(res.find(" cdr interval 600"), -1)
+        self.assertTrue(res.find(" cdr interval 900") > 0)
+        self.assertEqual(res.find(" cdr interval 600"), -1)
 
     def testVtyTimers(self):
         self.vty.enable()
         self.assertTrue(self.vty.verify('configure terminal', ['']))
-        self.assertEquals(self.vty.node(), 'config')
+        self.assertEqual(self.vty.node(), 'config')
         self.assertTrue(self.vty.verify('sgsn', ['']))
-        self.assertEquals(self.vty.node(), 'config-sgsn')
+        self.assertEqual(self.vty.node(), 'config-sgsn')
 
         for t in [3312, 3322, 3350, 3360, 3370, 3313, 3314, 3316, 3385, 3395, 3397]:
             self.assertTrue(self.vty.verify('timer t%d 10' % t, ['']))
@@ -316,9 +316,9 @@ if __name__ == '__main__':
     if args.p:
         confpath = args.p
 
-    print "confpath %s, workdir %s" % (confpath, workdir)
+    print("confpath %s, workdir %s" % (confpath, workdir))
     os.chdir(workdir)
-    print "Running tests for specific VTY commands"
+    print("Running tests for specific VTY commands")
     suite = unittest.TestSuite()
     add_gbproxy_test(suite, workdir)
     add_sgsn_test(suite, workdir)