RSGISLib Utility Tools

Unique Value

rsgislib.tools.utils.uid_generator(size: int = 6) str

A function which will generate a ‘random’ string of the specified length based on the UUID

Parameters

size – the length of the returned string.

Returns

string of length size.

Text File I/O

rsgislib.tools.utils.read_text_file_no_new_lines(input_file: str) str

Read a text file into a single string removing new lines.

Parameters

input_file – File path to the input file.

Returns

string

rsgislib.tools.utils.read_text_file_to_list(input_file: str, include_empty_lines: bool = False) List

Read a text file into a list where each line is an element in the list.

Parameters
  • input_file – File path to the input file.

  • include_empty_lines – Include empty lines within the line (Default: False)

Returns

list

rsgislib.tools.utils.write_list_to_file(data_lst: List, out_file: str)

Write a list a text file, one line per item.

Parameters
  • data_lst – List of values to be written to the output file.

  • out_file – File path to the output file.

rsgislib.tools.utils.write_data_to_file(data_val, out_file: str)

Write some data (a string or can be converted to a string using str(data_val) to an output text file.

Parameters
  • data_val – Data to be written to the output file.

  • out_file – File path to the output file.

rsgislib.tools.utils.create_ascii_text_file(input_file, output_file)

A function which will ensure that an input text file will only have ascii characters. Non-ascii characters will be removed.

Parameters
  • input_file – input file path

  • output_file – output file path

JSON File I/O

rsgislib.tools.utils.write_dict_to_json(data_dict: dict, out_file: str)

Write some data to a JSON file. The data would commonly be structured as a dict but could also be a list.

Parameters
  • data_dict – The dict (or list) to be written to the output JSON file.

  • out_file – The file path to the output file.

rsgislib.tools.utils.write_dict_to_json_gz(data_dict: dict, out_file: str, encoding: str = 'utf-8')

Write some data to a JSON file compressed with gzip. The data would commonly be structured as a dict but could also be a list.

Parameters
  • data_dict – The dict (or list) to be written to the output JSON file.

  • out_file – The file path to the output file.

  • encoding – the character set encoding (default: utf-8)

rsgislib.tools.utils.read_json_to_dict(input_file: str) Dict

Read a JSON file. Will return a list or dict.

Parameters

input_file – input JSON file path.

rsgislib.tools.utils.read_gz_json_to_dict(input_file: str, encoding: str = 'utf-8') Dict

Read a JSON file. Will return a list or dict.

Parameters
  • input_file – input JSON file path.

  • encoding – the character set encoding (default: utf-8)

Parse Dict/JSON

rsgislib.tools.utils.dict_struct_does_path_exist(dict_struct_obj: dict, tree_sequence: List) bool

A function which tests whether a path exists within JSON file.

Parameters
  • dict_struct_obj – the structure of dicts (i.e., JSON) to be searched

  • tree_sequence – list of strings specifying the path within the structure

Returns

boolean

rsgislib.tools.utils.dict_struct_get_str_value(dict_struct_obj: dict, tree_sequence: List, valid_values: Optional[List] = None) str

A function which retrieves a single string value from a JSON structure.

Parameters
  • dict_struct_obj – the structure of dicts (i.e., JSON) to be searched

  • tree_sequence – list of strings specifying the path within the structure

  • valid_values – An optional list of valid values. An exception will be thrown if a list is provided and the string value found is not within the list.

Returns

the string value at the path provided.

rsgislib.tools.utils.dict_struct_get_boolean_value(dict_struct_obj: dict, tree_sequence: List) bool

A function which retrieves a single boolean value from a JSON structure.

Parameters
  • dict_struct_obj – the structure of dicts (i.e., JSON) to be searched

  • tree_sequence – list of strings specifying the path within the structure

Returns

boolean value.

rsgislib.tools.utils.dict_struct_get_date_value(dict_struct_obj: dict, tree_sequence: List, date_format: str = '%Y-%m-%d') datetime.datetime

A function which retrieves a single date value from a JSON structure.

Parameters
  • dict_struct_obj – the structure of dicts (i.e., JSON) to be searched

  • tree_sequence – list of strings specifying the path within the structure

  • date_format – a string or list of strings for the date/time format to be parsed by datetime.datetime.strptime.

Returns

a date object with the value at the path specified.

rsgislib.tools.utils.dict_struct_get_datetime_value(dict_struct_obj: dict, tree_sequence: List, date_time_format: str = '%Y-%m-%dT%H:%M:%S.%f')

A function which retrieves a single date value from a JSON structure.

Parameters
  • dict_struct_obj – the structure of dicts (i.e., JSON) to be searched

  • tree_sequence – list of strings specifying the path within the structure

  • date_time_format – a string or list of strings for the date/time format to be parsed by datetime.datetime.strptime.

Returns

datetime object with the value at the path provided.

rsgislib.tools.utils.dict_struct_get_str_list_value(dict_struct_obj: dict, tree_sequence: List, valid_values: Optional[List] = None) List[str]

A function which retrieves a list of string values from a JSON structure.

Parameters
  • dict_struct_obj – the structure of dicts (i.e., JSON) to be searched

  • tree_sequence – list of strings specifying the path within the structure

  • valid_values – An optional list of valid values. An exception will be thrown if a list is provided and a string value found is not within the list.

Returns

list of strings

rsgislib.tools.utils.dict_struct_get_numeric_value(dict_struct_obj: dict, tree_sequence: List, valid_lower: Optional[float] = None, valid_upper: Optional[float] = None) Union[int, float]

A function which retrieves a single numeric value from a JSON structure.

Parameters
  • dict_struct_obj – the structure of dicts (i.e., JSON) to be searched

  • tree_sequence – list of strings specifying the path within the structure

  • valid_lower – optional lower valid bounds, error throw if outside the bounds.

  • valid_upper – optional upper valid bounds, error throw if outside the bounds.

Returns

numeric value at path

rsgislib.tools.utils.dict_struct_get_list_value(dict_struct_obj: dict, tree_sequence: List) List

A function which retrieves a list of values from a JSON structure.

Parameters
  • dict_struct_obj – the structure of dicts (i.e., JSON) to be searched

  • tree_sequence – list of strings specifying the path within the structure

Returns

list of values

Strings

rsgislib.tools.utils.zero_pad_num_str(num_val: float, str_len: int = 3, round_num: bool = False, round_n_digts: int = 0, integerise: bool = False, absolute: bool = False, gain: float = 1) str

A function which zero pads a number to make a string

Parameters
  • num_val – number value to be processed.

  • str_len – the number of characters in the output string.

  • round_num – boolean whether to round the input number value.

  • round_n_digts – If rounding, the number of digits following decimal points to round to.

  • integerise – boolean whether to integerise the input number

  • absolute – make number positive.

  • gain – apply a gain before integerising.

Returns

string with the padded numeric value.

rsgislib.tools.utils.remove_repeated_chars(str_val: str, repeat_char: str) str

A function which removes repeated characters within a string for the specified character

Parameters
  • str_val – The input string.

  • repeat_char – The character

Returns

string without repeat_char

rsgislib.tools.utils.check_str(str_val: str, rm_non_ascii: bool = False, rm_dashs: bool = False, rm_spaces: bool = False, rm_punc: bool = False) str

A function which can check a string removing spaces (replaced with underscores), remove punctuation and any non ascii characters.

Parameters
  • str_val – the input string to be processed.

  • rm_non_ascii – If True (default False) remove any non-ascii characters from the string

  • rm_dashs – If True (default False) remove any dashs from the string and replace with underscores.

  • rm_spaces – If True (default False) remove any spaces from the string.

  • rm_punc – If True (default False) remove any punctuation (other than ‘_’ or ‘-‘) from the string.

Returns

returns a string outputted from the processing.

Encode String

rsgislib.tools.utils.encode_base64_text(input_txt: str) str

A function which encoded the input text using base64 into bytes. Useful for storing passwords in configure files, which while not secure means they cannot just be read as plain text.

Parameters

plain_txt – The input string to be encoded

Returns

Output encoded string.

rsgislib.tools.utils.decode_base64_text(in_encoded_txt: str) str

A function which decoded text encoded using the encode_base64_text function.

Parameters

in_encoded_txt – Input encoded byte string (i.e., encoded using the encode_base64_text function).

Returns

Output plain text string

rsgislib.tools.utils.create_username_password_file(username: str, password: str, out_file: str)

A function which will create a username/password file where the username and password are encoded using base64 so the information is not stored in plain text - note this is note ‘secure’ just means that someone cannot just read the plain text username and password.

Parameters
  • username – string with the username

  • password – string with the password

  • out_file – output file path.

rsgislib.tools.utils.get_username_password(input_file: str) -> (<class 'str'>, <class 'str'>)

A function which retrieves a username and password from an input file where the data has been encoded with base64.

Parameters

input_file – input file path.

Returns

plain text username and password

Numeric

rsgislib.tools.utils.is_number(str_val: str) bool

A function which tests whether the input string contains a number of not.

Returns

boolean

rsgislib.tools.utils.in_bounds(x: numpy.array, lower: float, upper: float, upper_strict: bool = False) bool

Checks whether a value or array of values is within specified bounds.

Parameters
  • x – value or numpy array of values to check.

  • lower – lower bound

  • upper – upper bound

  • upper_strict – True is less than upper; False is less than equal to upper

Returns

boolean

rsgislib.tools.utils.mixed_signs(x: numpy.array) bool

Check whether an array of numbers has a mix of positive and negative values.

Parameters

x – list of values.

Returns

boolean

rsgislib.tools.utils.negative(x: numpy.array) bool

Is the maximum number in the list negative. :param x: list of values

Returns

boolean

rsgislib.tools.utils.is_odd(number: float) bool

A function which tests whether a number is odd

Parameters

number – number value to test.

Returns

True = input number is odd; False = input number is even

Colours

rsgislib.tools.utils.hex_to_rgb(hex_str: str) -> (<class 'int'>, <class 'int'>, <class 'int'>)

A function which converts an hexadecimal colour representation to RGB values between 0 and 255.

For example: #b432be is equal to: 180, 50, 190

Parameters

hex_str – Input hex string which can be either 7 or 6 characters long. If 7 characters then the first character will be a #.

Returns

R, G, B tuple

import rsgislib.tools.utils
r, g, b = rsgislib.tools.utils.hex_to_rgb("#b432be")
rsgislib.tools.utils.rgb_to_hex(r: Union[int, float], g: Union[int, float], b: Union[int, float], normalised: bool = False) str

A function which converts red, green, blue values to a hexadecimal colour representation.

For example: 180, 50, 190 is equal to: #b432be

Parameters
  • r – number with range either 0-255 or 0-1 if normalised

  • g – number with range either 0-255 or 0-1 if normalised

  • b – number with range either 0-255 or 0-1 if normalised

  • normalised – a boolean specifying the inputs are in range 0-1

Returns

string with hexadecimal colour representation

Dates

rsgislib.tools.utils.get_days_since(year: int, day_of_year: int, base_date: datetime.date) int

Calculate the number of days from a base data to a defined year/day.

Parameters
  • year – int with year XXXX (e.g., 2020)

  • day_of_year – int with the day within the year (1-365)

  • base_date – a datetime.date object

Returns

int (n days)

rsgislib.tools.utils.get_days_since_date(year: int, month: int, day, base_date: datetime.date) int

Calculate the number of days from a base data to a defined year/day.

Parameters
  • year – int with year XXXX (e.g., 2020)

  • month – int month in year (1-12) (e.g., 6)

  • day – int with the day within the month (1-31) (e.g., 20)

  • base_date – a datetime.date object

Returns

int (n days)

Powerset

rsgislib.tools.utils.powerset_iter(in_set: List)

A function which returns an iterator (generator) for all the subsets of the inputted set (i.e., the powerset)

Params inset

the input set for which the powerset will be produced

rsgislib.tools.utils.powerset_lst(in_set: List, min_items: int = 0) List

A function which returns a list for all the subsets of the inputted set (i.e., the powerset)

Params inset

the input set for which the powerset will be produced

Params min_items

Optional parameter specifying the minimum number of items in the output sets. If 0 or below then ignored. Default is 0.

rsgislib.tools.utils.create_var_list(in_vals_lsts: dict, val_dict: Optional[dict] = None) List[Dict]

A function which will produce a list of dictionaries with all the combinations of the input variables listed (i.e., the powerset).

Parameters
  • in_vals_lsts – dictionary with each value having a list of values.

  • val_dict – variable used in iterative nature of function which lists the variable for which are still to be looped through. Would normally not be provided by the user as default is None. Be careful if you set as otherwise you might not get a correct or valid result.

Returns

list of dictionaries with the same keys are the input but only a single value will be associate with key rather than a list.

import rsgislib.tools.utils
seg_vars_ranges = dict()
seg_vars_ranges['k'] = [5, 10, 20, 30, 40, 50, 60, 80, 100, 120]
seg_vars_ranges['d'] = [10, 20, 50, 100, 200, 1000, 10000]
seg_vars_ranges['minsize'] = [5, 10, 20, 50, 100, 200]
seg_vars = rsgislib.tools.utils.create_var_list(seg_vars_ranges)

Compute Environment

rsgislib.tools.utils.get_environment_variable(var: str) str

A function to get an environmental variable, if variable is not present returns None.

Parameters

var – the name of the environmental variable to get the value for.

Returns

value of env var.

rsgislib.tools.utils.num_process_cores() int

A functions which returns the number of processing cores available on the machine

Returns

int