RSGISLib Utility Tools

Unique Value

rsgislib.tools.utils.uid_generator(size: int = 6) → str

A function which will generate a ‘random’ string of the specified length based on the UUID

Parameters:

size – the length of the returned string.

Returns:

string of length size.

Text File I/O

rsgislib.tools.utils.read_text_file(input_file: str) → str

Read a text file into a single string.

Parameters:

input_file – File path to the input file.

Returns:

string

rsgislib.tools.utils.read_text_file_no_new_lines(input_file: str) → str

Read a text file into a single string removing new lines.

Parameters:

input_file – File path to the input file.

Returns:

string

rsgislib.tools.utils.read_text_file_to_list(input_file: str, include_empty_lines: bool = False) → List

Read a text file into a list where each line is an element in the list.

Parameters:

  • input_file – File path to the input file.

  • include_empty_lines – Include empty lines within the line (Default: False)

Returns:

list

rsgislib.tools.utils.write_list_to_file(data_lst: List, out_file: str)

Write a list a text file, one line per item.

Parameters:

  • data_lst – List of values to be written to the output file.

  • out_file – File path to the output file.

rsgislib.tools.utils.write_data_to_file(data_val, out_file: str)

Write some data (a string or can be converted to a string using str(data_val) to an output text file.

Parameters:

  • data_val – Data to be written to the output file.

  • out_file – File path to the output file.

rsgislib.tools.utils.create_ascii_text_file(input_file: str, output_file: str)

A function which will ensure that an input text file will only have ascii characters. Non-ascii characters will be removed.

Parameters:

  • input_file – input file path

  • output_file – output file path

rsgislib.tools.utils.prettify_xml_file(input_file: str, output_file: str)

A function which prettifies an XML file - i.e., inserts new lines and tabs.

Parameters:

  • input_file – The input file path.

  • output_file – The output file path.

JSON File I/O

rsgislib.tools.utils.write_dict_to_json(data_dict: Dict, out_file: str)

Write some data to a JSON file. The data would commonly be structured as a dict but could also be a list.

Parameters:

  • data_dict – The dict (or list) to be written to the output JSON file.

  • out_file – The file path to the output file.

rsgislib.tools.utils.write_dict_to_json_gz(data_dict: Dict, out_file: str, encoding: str = 'utf-8')

Write some data to a JSON file compressed with gzip. The data would commonly be structured as a dict but could also be a list.

Parameters:

  • data_dict – The dict (or list) to be written to the output JSON file.

  • out_file – The file path to the output file.

  • encoding – the character set encoding (default: utf-8)

rsgislib.tools.utils.read_json_to_dict(input_file: str) → Dict

Read a JSON file. Will return a list or dict.

Parameters:

input_file – input JSON file path.

rsgislib.tools.utils.read_gz_json_to_dict(input_file: str, encoding: str = 'utf-8') → Dict

Read a JSON file. Will return a list or dict.

Parameters:

  • input_file – input JSON file path.

  • encoding – the character set encoding (default: utf-8)

Parse Dict/JSON

rsgislib.tools.utils.dict_to_json_str(data_dict: Dict) → str

Write some data to a JSON file. The data would commonly be structured as a dict but could also be a list.

Parameters:

  • data_dict – The dict (or list) to be written to the output JSON file.

  • out_file – The file path to the output file.

rsgislib.tools.utils.dict_struct_does_path_exist(dict_struct_obj: Dict, tree_sequence: List) → bool

A function which tests whether a path exists within JSON file.

Parameters:

  • dict_struct_obj – the structure of dicts (i.e., JSON) to be searched

  • tree_sequence – list of strings specifying the path within the structure

Returns:

boolean

rsgislib.tools.utils.dict_struct_get_str_value(dict_struct_obj: Dict, tree_sequence: List, valid_values: List = None) → str

A function which retrieves a single string value from a JSON structure.

Parameters:

  • dict_struct_obj – the structure of dicts (i.e., JSON) to be searched

  • tree_sequence – list of strings specifying the path within the structure

  • valid_values – An optional list of valid values. An exception will be thrown if a list is provided and the string value found is not within the list.

Returns:

the string value at the path provided.

rsgislib.tools.utils.dict_struct_get_boolean_value(dict_struct_obj: Dict, tree_sequence: List) → bool

A function which retrieves a single boolean value from a JSON structure.

Parameters:

  • dict_struct_obj – the structure of dicts (i.e., JSON) to be searched

  • tree_sequence – list of strings specifying the path within the structure

Returns:

boolean value.

rsgislib.tools.utils.dict_struct_get_date_value(dict_struct_obj: Dict, tree_sequence: List, date_format: str = '%Y-%m-%d') → datetime

A function which retrieves a single date value from a JSON structure.

Parameters:

  • dict_struct_obj – the structure of dicts (i.e., JSON) to be searched

  • tree_sequence – list of strings specifying the path within the structure

  • date_format – a string or list of strings for the date/time format to be parsed by datetime.datetime.strptime.

Returns:

a date object with the value at the path specified.

rsgislib.tools.utils.dict_struct_get_datetime_value(dict_struct_obj: Dict, tree_sequence: List, date_time_format: str = '%Y-%m-%dT%H:%M:%S.%f')

A function which retrieves a single date value from a JSON structure.

Parameters:

  • dict_struct_obj – the structure of dicts (i.e., JSON) to be searched

  • tree_sequence – list of strings specifying the path within the structure

  • date_time_format – a string or list of strings for the date/time format to be parsed by datetime.datetime.strptime.

Returns:

datetime object with the value at the path provided.

rsgislib.tools.utils.dict_struct_get_str_list_value(dict_struct_obj: Dict, tree_sequence: List, valid_values: List = None) → List[str]

A function which retrieves a list of string values from a JSON structure.

Parameters:

  • dict_struct_obj – the structure of dicts (i.e., JSON) to be searched

  • tree_sequence – list of strings specifying the path within the structure

  • valid_values – An optional list of valid values. An exception will be thrown if a list is provided and a string value found is not within the list.

Returns:

list of strings

rsgislib.tools.utils.dict_struct_get_numeric_value(dict_struct_obj: Dict, tree_sequence: List, valid_lower: float = None, valid_upper: float = None) → int | float

A function which retrieves a single numeric value from a JSON structure.

Parameters:

  • dict_struct_obj – the structure of dicts (i.e., JSON) to be searched

  • tree_sequence – list of strings specifying the path within the structure

  • valid_lower – optional lower valid bounds, error throw if outside the bounds.

  • valid_upper – optional upper valid bounds, error throw if outside the bounds.

Returns:

numeric value at path

rsgislib.tools.utils.dict_struct_get_list_value(dict_struct_obj: Dict, tree_sequence: List) → List

A function which retrieves a list of values from a JSON structure.

Parameters:

  • dict_struct_obj – the structure of dicts (i.e., JSON) to be searched

  • tree_sequence – list of strings specifying the path within the structure

Returns:

list of values

Strings

rsgislib.tools.utils.zero_pad_num_str(num_val: float, str_len: int = 3, round_num: bool = False, round_n_digts: int = 0, integerise: bool = False, absolute: bool = False, gain: float = 1) → str

A function which zero pads a number to make a string

Parameters:

  • num_val – number value to be processed.

  • str_len – the number of characters in the output string.

  • round_num – boolean whether to round the input number value.

  • round_n_digts – If rounding, the number of digits following decimal points to round to.

  • integerise – boolean whether to integerise the input number

  • absolute – make number positive.

  • gain – apply a gain before integerising.

Returns:

string with the padded numeric value.

rsgislib.tools.utils.remove_repeated_chars(str_val: str, repeat_char: str) → str

A function which removes repeated characters within a string for the specified character

Parameters:

  • str_val – The input string.

  • repeat_char – The character

Returns:

string without repeat_char

rsgislib.tools.utils.check_str(str_val: str, rm_non_ascii: bool = False, rm_dashs: bool = False, rm_spaces: bool = False, rm_punc: bool = False) → str

A function which can check a string removing spaces (replaced with underscores), remove punctuation and any non ascii characters.

Parameters:

  • str_val – the input string to be processed.

  • rm_non_ascii – If True (default False) remove any non-ascii characters from the string

  • rm_dashs – If True (default False) remove any dashes from the string and replace with underscores.

  • rm_spaces – If True (default False) remove any spaces from the string.

  • rm_punc – If True (default False) remove any punctuation (other than ‘_’ or ‘-’) from the string.

Returns:

returns a string outputted from the processing.

Encode String

rsgislib.tools.utils.encode_base64_text(input_txt: str) → str

A function which encoded the input text using base64 into bytes. Useful for storing passwords in configure files, which while not secure means they cannot just be read as plain text.

Parameters:

plain_txt – The input string to be encoded

Returns:

Output encoded string.

rsgislib.tools.utils.decode_base64_text(in_encoded_txt: str) → str

A function which decoded text encoded using the encode_base64_text function.

Parameters:

in_encoded_txt – Input encoded byte string (i.e., encoded using the encode_base64_text function).

Returns:

Output plain text string

rsgislib.tools.utils.create_username_password_file(username: str, password: str, out_file: str)

A function which will create a username/password file where the username and password are encoded using base64 so the information is not stored in plain text - note this is note ‘secure’ just means that someone cannot just read the plain text username and password.

Parameters:

  • username – string with the username

  • password – string with the password

  • out_file – output file path.

rsgislib.tools.utils.get_username_password(input_file: str) -> (<class 'str'>, <class 'str'>)

A function which retrieves a username and password from an input file where the data has been encoded with base64.

Parameters:

input_file – input file path.

Returns:

plain text username and password

Numeric

rsgislib.tools.utils.is_number(str_val: str) → bool

A function which tests whether the input string contains a number of not.

Returns:

boolean

rsgislib.tools.utils.in_bounds(x: array, lower: float, upper: float, upper_strict: bool = False) → bool

Checks whether a value or array of values is within specified bounds.

Parameters:

  • x – value or numpy array of values to check.

  • lower – lower bound

  • upper – upper bound

  • upper_strict – True is less than upper; False is less than equal to upper

Returns:

boolean

rsgislib.tools.utils.mixed_signs(x: array) → bool

Check whether an array of numbers has a mix of positive and negative values.

Parameters:

x – list of values.

Returns:

boolean

rsgislib.tools.utils.negative(x: array) → bool

Is the maximum number in the list negative. :param x: list of values

Returns:

boolean

rsgislib.tools.utils.is_odd(number: float) → bool

A function which tests whether a number is odd

Parameters:

number – number value to test.

Returns:

True = input number is odd; False = input number is even

Colours

rsgislib.tools.utils.hex_to_rgb(hex_str: str) -> (<class 'int'>, <class 'int'>, <class 'int'>)

A function which converts an hexadecimal colour representation to RGB values between 0 and 255.

For example: #b432be is equal to: 180, 50, 190

Parameters:

hex_str – Input hex string which can be either 7 or 6 characters long. If 7 characters then the first character will be a #.

Returns:

R, G, B tuple

import rsgislib.tools.utils
r, g, b = rsgislib.tools.utils.hex_to_rgb("#b432be")
rsgislib.tools.utils.rgb_to_hex(r: int | float, g: int | float, b: int | float, normalised: bool = False) → str

A function which converts red, green, blue values to a hexadecimal colour representation.

For example: 180, 50, 190 is equal to: #b432be

Parameters:

  • r – number with range either 0-255 or 0-1 if normalised

  • g – number with range either 0-255 or 0-1 if normalised

  • b – number with range either 0-255 or 0-1 if normalised

  • normalised – a boolean specifying the inputs are in range 0-1

Returns:

string with hexadecimal colour representation

Dates

rsgislib.tools.utils.get_days_since(year: int, day_of_year: int, base_date: date) → int

Calculate the number of days from a base data to a defined year/day.

Parameters:

  • year – int with year XXXX (e.g., 2020)

  • day_of_year – int with the day within the year (1-365)

  • base_date – a datetime.date object

Returns:

int (n days)

rsgislib.tools.utils.get_days_since_date(year: int, month: int, day, base_date: date) → int

Calculate the number of days from a base data to a defined year/day.

Parameters:

  • year – int with year XXXX (e.g., 2020)

  • month – int month in year (1-12) (e.g., 6)

  • day – int with the day within the month (1-31) (e.g., 20)

  • base_date – a datetime.date object

Returns:

int (n days)

rsgislib.tools.utils.find_month_end_date(year: int, month: int) → int

A function which returns the date of the last day of the month.

Parameters:

  • year – int for the year (e.g., 2019)

  • month – month (e.g., 9)

Returns:

last day of the month date

rsgislib.tools.utils.get_datetime_increment_lst(start_date: datetime, end_date: datetime, interval: timedelta) → List[datetime]

A function which returns a list of datetime objects from the start date in steps of the interval until the date is greater than the end date.

Parameters:

  • start_date – the start datetime of the sequence

  • end_date – the end datetime of the sequence

  • interval – the interval as a timedelta object.

Returns:

List of datetime objects

rsgislib.tools.utils.is_summer_winter(lat: float, date: date) → int

A function which returns an integer specifying whether the date provided is within the summer or winter for a given latitude.

Southern Hemisphere Winter: May - November Northern Hemisphere Winter: March - October

Parameters:

  • lat – float for the latitude

  • date – a datetime.date object

Returns:

int (1 == summer; 2 == winter)

rsgislib.tools.utils.create_year_month_start_end_lst(start_year: int, start_month: int, end_year: int, end_month: int) → List[Tuple[int, int]]

A function which creates a list of year and month tuples from a start and end month and year.

Parameters:

  • start_year – int with the start year

  • start_month – int with the start month

  • end_year – int with the end year

  • end_month – int with the end month

Returns:

List of tuples (year, month)

rsgislib.tools.utils.create_year_month_n_months_lst(start_year: int, start_month: int, n_months: int) → List[Tuple[int, int]]

A function which creates a list of year and month tuples from a start and end month and year.

Parameters:

  • start_year – int with the start year

  • start_month – int with the start month

  • n_months – int with the number of months ahead

Returns:

List of tuples (year, month)

Powerset

rsgislib.tools.utils.powerset_iter(in_set: List)

A function which returns an iterator (generator) for all the subsets of the inputted set (i.e., the powerset)

Params inset:

the input set for which the powerset will be produced

rsgislib.tools.utils.powerset_lst(in_set: List, min_items: int = 0) → List

A function which returns a list for all the subsets of the inputted set (i.e., the powerset)

Params inset:

the input set for which the powerset will be produced

Params min_items:

Optional parameter specifying the minimum number of items in the output sets. If 0 or below then ignored. Default is 0.

rsgislib.tools.utils.create_var_list(in_vals_lsts: Dict, val_dict: Dict = None) → List[Dict]

A function which will produce a list of dictionaries with all the combinations of the input variables listed (i.e., the powerset).

Parameters:

  • in_vals_lsts – dictionary with each value having a list of values.

  • val_dict – variable used in iterative nature of function which lists the variable for which are still to be looped through. Would normally not be provided by the user as default is None. Be careful if you set as otherwise you might not get a correct or valid result.

Returns:

list of dictionaries with the same keys are the input but only a single value will be associate with key rather than a list.

import rsgislib.tools.utils
seg_vars_ranges = dict()
seg_vars_ranges['k'] = [5, 10, 20, 30, 40, 50, 60, 80, 100, 120]
seg_vars_ranges['d'] = [10, 20, 50, 100, 200, 1000, 10000]
seg_vars_ranges['minsize'] = [5, 10, 20, 50, 100, 200]
seg_vars = rsgislib.tools.utils.create_var_list(seg_vars_ranges)

Compute Environment

rsgislib.tools.utils.get_environment_variable(var: str) → str

A function to get an environmental variable, if variable is not present returns None.

Parameters:

var – the name of the environmental variable to get the value for.

Returns:

value of env var.

rsgislib.tools.utils.num_process_cores() → int

A functions which returns the number of processing cores available on the machine

Returns:

int