How to Tokenize String in Golang

Tokenizing a string involves splitting a string into multiple parts called tokens. In the Golang, you can easily tokenize strings using the built-in strings.Split() function from the strings package.

Syntax

func Split(s, sep string) []string

Arguments:

• s = the string to be tokenized
• sep = the delimiter (separator) to split the string by.

Return Value:

• Returns string array containing the individual tokens after splitting the original string.

In this article, we will discuss how to use the strings.Split() function in golang to tokenize a string into tokens by a separator.

How to Tokenize String Using strings.Split() in Golang

To tokenize a string using the strings.Split() function in Golang, pass the two arguments to the function; a string and a delimiter to split a string. This function returns a string array.

package main

import (
	"fmt"
	"strings"
)

func main() {

	// string tokenizer
	s := "192.168.0.1:5001"

        // Split string by : using the Split() function
        // stores the substrings in tokens variable

	tokens := strings.Split(s, ":")

        // Print the token 0 or substring at index 0

	fmt.Println("Token 0:", tokens[0]) // Output: 192.168.0.1
	fmt.Println("Token 1:", tokens[1]) // Output: 5001

}
                  

The above golang program imports strings package to use Split function and fmt package.

Golang strings Split function takes 2 arguments:

1. string as “192.168.0.1:5001”
2. delimiter as a colon: to break the string into tokens

In the first example, strings.Split() function splits a string by a character colon (:) and returns a slice of substrings ( string array).

The output of the above golang tokenizes a string program to break a string into multiple parts:

The output of the above golang program to split a string by character is:

Token 0: 192.168.0.1
Token 1: 5001

Cool Tip: How to use strings.SplitN() function in Golang!

Conclusion

I hope the above article on how to use the Split() function in golang to tokenize a string is helpful to you.

The strings.Split() function is case-sensitive. If the separator string is empty, the function returns a slice containing the original string as the only element.

You can find more topics about the Golang tutorials on the GolangSpot Home page.